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ABSTRACT 

The 12 studies that constituted the 1971-1972 
evaluation of the Let's Be Amigos program are presented together with 
the evaluation findings. The three components of the Bilingual 
Program, serving 1,813 students by the end of the third year, were 
the. Model A program (prekindergarten to grade 3) with teams of 
English- and Spanish-speaking teachers working with the students; the 
Hodel B program (grades 1 and 2) in which teachers worked with 
students who were dominant in the teacher's mother tongue; and the 
ARRIBA program (elementary, junior high, and senior high schools) 
serving primarily Latino pupils who could benefit from instruction in 
their mother tongue. The evaluation studies were categorized as to 
Process Evaluation Studies (four studies) and Product Evaluation 
Studies ([eight studies). The Process Evaluation Studies were: General 
Process Evaluation of the Third Year of the Bilingual Program; 
Perceptions of Prinicpals, Teachers, and Parents Regarding the 
Bilingual Program; Curriculum Development, Distribution, ancl Tryout 
in the ARRIBA Program; and Follow-up of Elementary- Level ARRIBA 
Pupils Who Moved into Regular English Language Classes. The Product 
Evaluation Studies were: Log of Pupil Performance in the Model A and 
Hodel B Programs; Criterion-Referent Testing of Second- and 
Third-Grade Arithmetic Skills in the Hodel School Program; 
Exploratory Testing of ARRIBA Junior High School Students; and 
Replication of Findings Assessing the ARRIBA Component's Impact on 
Elementary School Pupils* Attendance and High School Pupils' Dropout 
Rate. (DB) 
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. SUMMARY OF THE^- "LET ' S HJr A.XrGOS " PKOJSCT EVALUATION 



' - \ ■ The Program . 

By the' end of the third operational year, the three coinponej:its — ' 
of the Bilingual Prograra were serving 1,813 students — 937 in' the Model A 
Bilingual School, 277 in the Model B Bilingual School programs , and 599 
in ARRIBA programs. Of these students ,1,005 were judged to be "Spanish- . 
speaking, • 172 were judged to be speakers of both Spanish and English, 
and 527 were judged to be speakers of English. (Data are lacking for 
the remaining few students)'. The English-speaking pupils were virtually 

■ all participants in the Model A and Model B schools.. ARRIBA students 
were- nearly all Spanish-speaking. - " 

The Model A program operated in the Potter-Thomas School pre- 
kindergarten to third grade,' with teams of English- and Spanish -speaking 
teachers, working with the pupils. In the earlier 'grade levels instruc- 
tion was predominantly in the mother tongue. As pupils matured, increased . 
contact with the second language was built into the program. 

'Ihree schools — Miller,. Ludlow, and Bethune — ^had first and 'second 
grade classes which comprised the Model B component. The course of "study 
•was similar to that in Model A but the staffing -pattern was different. 
English- and Spanish -speaking teachers worked with classes of pupils 
dominant in the teacher's mother tongue." Itinerant second-language 
specialists worked with the pupils in their* specialties- — Englisli as a 
Second' Language and Spanish. as a Second Language. 

Eight schools— two elementary, two junior high, and four senior 
high schools — comprised the. ARRIBA program. This sejrved primarily Latino 
.pupils who could benefit from, or wanted instruction in, ^ their- mother 

■ tongue . At all ' levels courses were of fered in Spanish as a First Lan- 
guage, English as a Second Language , Social ^Jtudies in Suahish , and Science 
in, Spanish. Thie content of the courses varied with" grade level and was 
.(where possiblie) similar to the English-language, program studied in the 
Philadelphia schools. At the elementary and junior high schools pupils 
served by the program took all courses offered for their grade levels. 

At th,e high school level pupilis • could select courses from among those 
offered by the program. In one high, school Kensington , some commercial 
courses in Spanish were added to the ARRIBA component during the" 1971-1972 
year. . c» . 

Program Costs . 

Table /o.l shows grant funds* expenditures per pupil and for 
each of' the .-project, components in the Let-s Be Amigos project. Evaluation 
and audit costs were excluded since expenditures for these services would 
not be required if. the program were . not funded under ESEA Title VII. The 
direct expenditures at program sites" were those expenditures which could 
be clearly assigned to various program coirponents — mostly for- supervision 
and faculty- who. taught and implemented the materials which had been devel- 
oped for^ the program*'. The prorated costs are those project expenses which 



TABLE 0.1 



Expenditure of Grant Funds for Various Components of the 
Program for Services to ChildrcHr (Excluding the 
Costs of Evaluation- and Audit) 



Program 


Direct 
Expenditures 
at Program 
Sites 


Prorated 
Costs of 
Centralized 
Activities 


Total* 


Model A (937 pupils) 
Total 

Cost per pupil 


$163,861.43 
174.88 


$ 57,417.79 
61.28 


$221,279.22 
236.16 


Model B (277 DUDils) 
Total 

Cost per pupil 


39,830.30 
143.79 


16,974.10 
61.28 


56,804.40 
205.07 


ARRIBA (599 pupils) 
Total 

Cost per pupil 


172,841.95 
288,55 


36,705.72 
61.28 


209,547.67 
349.83 


Entire Program (1813 pupils) 
Total 

Cost per pupil 


376, 533.68 
207.69 


111,097.61 
61.28 


487,631.29 
268.9? 

J", 



♦Excluding evaluation and audit costs. 



could not easily be assigned to specific components. They included text 
purchases, curriculum-development activities, project mcmagement , and 
public relations activities. The variation in per-pupil cost among the 
three components is due primarily to the variation in proportion of 
teaching staff which was paid out of School District operating funds. 

The 1970-1971 Evaluation reported that the overall cost from 
>itle VII funds was $353.37 per pupil. This cost included about" $13.00 
per pupil for evaluation making the net cost per pupil about $340. During 
the 1971-1972 year, the operational cost from Title VII (exclusive of 
evaluation) was $269. This is a reduction of 21%. This cost reduction 
was brought about by vertical expansion of the Model A and Model B programs, 
and increase in the number of pupils served in the ARRIBA component , 
expansion which was paid for primarily with School District operating funds. 
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S Lurna r>^ o f Thi s Report 



Major Findings 

The major findings of the twelve studies which constitute the 
1971-1972 evaluation report show that the program was operating within 
the guidelines proposed for it^ and, vith sorp.e exceptions, achieving its 
major aims. 

Process Evaluation Studies 

Study 1 showed that groups made up of students from different \|, 
grade levels in the Model A program were established to accomodate pupils ^ 
entering in the second grades without prior' bilingual experience. It also' 
reported that the Model B component^ at the three schools in which it 
was operational, was in closer conformity to the guidelines than it had 
been in past years. The ARKIBA program expanded^ with increased service 
at the high school level as the major change: teaching staff and pupil 
participants increased at the high school level, and one school developed 
a selection of Spanish-* language commercial subjects. 

Study 2 reported that principals, teachers, and parents were 
supportive of the program. Eighty percent of parents asked that their 
children be continued in the project. Principals' and teachers' concerns 
centered on supervision and the availability of curriculum materials. - 

Study 3 examined curriculum development and curriculum distri- 
bution in the ARRIBA component. Findings were that progress in curriculum 
preparation was continuing to be made. Suggestions for improving the 
distribution of materials were gathered. 

Study 4 followed up ARRIBA elementary students who were trans- 
ferred to regular English-language classes. Results showed that older 
Spcinish-speaking pupils need several years of bilingual education before 
they. are ready to participate fully in regular English-language classes. 

P roduct Evaluation Studies 

Study 5 , Log of Pupil Performance in Models A and B, showed 
that pupils in the lower grades (prekindergarten , kindergarten, first 
grade) tended to meet project objectives by midyear. This suggests that 
these objectives could be upgraded to include more complex skills than 
they do now. In the upper grade levels pupils at midyear did not attain 
objectives, and there was little growth from midyear to year end. This 
suggests that teachers did not emphasize unmastered skills specified in 
the objectives during the latter half of the year. In Model B, there was 
generally more midyear-to-year-end growth than in Model A, suggesting 
that the problem may lie in Model A component management. 

S tudy 6 , examinina prekindergarten pupils' mastery of reading 
and nuiriber-readiness skills, suggested that there was a pool of pupils 
with cin adequate mastery of skills for an all-day kindergarten similar 
in size to the group in the program this year. The study also showed 
thet reading-- readiness skill items were mastered by more pupils than 
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were number-readiness skill items. 

Study 7 examined pupil performance on the Philadelphia Readiness 
Test. During the first year of operation, results were very good, with 
both groups exceeding the criteria of the program and Latino pupils tested 
in Spanish scoring higher ^than pupils in any other school. In the second 
operational year. Latino pupil performance had fallen below the objective- 
specified base lines'. In 1971-1972 puoil performance on the Philadelphia 
Readiness Test rose close to the very high levels observed during the first 
operational year of the program. 

Study 8 reported pupils* performance on standardized tests. The 
performance of second-grade Anglo pupils in the program was superior at a 
statistically significant level to that of a base-line group of similar 
pupils on three of the four subtests of the Stanford Primary Battery II 
that were administered — Paragraph Meaning, Spelling, and Word Study 
Skills. Performances of Anglo pupils on the Word Meanina subtest were 
■ examined separately, for each of the Model School components. It was 
fpund that gains on the test were greater in Model A than in Model B, 
The gains in Model B over tJie base line were less than one month of grade- 
equivalent score. 

Third-grade Model A Anglos were examined on the Iowa Tests of Basic 
Skills . Comparison with the base line shov;ed a statistically significant 
improvement . in the Vocabulary subtest, marginally significant gains in 
Arithmetic and Language Skills,' and a gain, not st atistically significant,- 
in Reading. The composite score showed an overall advance of 2.5 months 
of grade-equivalent score. 

Examination of second-and third-grade Latinos on the Spanish reading 
test, Test De Destrezas Basicas En Lectura , showed strong, statistically 
significant gains over preprogram base lines — 13 percentile points in the second 
grade, and 17 percentile points in the third grade. Models A and B were about 
equally effective in teaching the reading skills. In the third grade, there 
was a trend (not statistically significant by the method of analysis used) 
for pupils who were in the bilingual program longer to earn higher scores than 
newly admitted pupils in the same grade level. 

Testing of third-grade Latino pupils, most of whom had received 
English reading instruction for about one year, showed that they read English 
at about the level of English-speaking pupils in a national sample at the 
end of first grade. 

Study 9 discusses performance on criterion-referent reading tests 
of first-grade pupils in their first language, second-grade Latinos in their 
second language (English) , and third-grade Anglos in their second language 
(Spanish) . Results showed that at most levels of the program fewer pupils 
could call words at levels specified in the objectives than had been anticipated 
in the proposal. 

Study 10 examined Model A and B second- and third-grade pupil 
performance on project-developed criterion-referent arithmetic tests. Results 
showed that pupil performance was similiar to that expected, with s^±>stantial 
numbers of pupils performing higher rhan project planners had anticipated in 
writing the, objectives . There were also a few pupils who. were below minimum 
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levels of performance specified by the project planners. 



Study 11 examined performance of junior nigh school ARRIBA pupils 
on standardized tests — the Inter-American Reading -Test in Spanish and the 
Iowa Test of Basic Skills in ^English. Seventli- and eighth-grade 
pupil performance on the Inter-American test showed significant gains over 
performance of similar pupils enrolled in the same grade levels prior to the 
initiation of the ARRIBA component in the 55chools. 

Administration of various levels of the Iowa Tests of Basic Skills 
showed that none of the levels in the battery is likely to be a suitable tool 
for evaluating emerging English skills of pupil participants. 

Study 12 replicated the finding of the 1970-1971 report that the high 
school lev^l of the ARRIBA program is effective in reducing the number of high 
school dropouts. This study also attempted to replicate the 1970-1971 finding 
that attendance of program participants at the upper elementary grade levels 
was improved significantly by participation in the ARRIBA program. Results 
showed that there was a trend for attendance to be improved, but it was not 
large enough to be statistically significant. 

Evaluation reports of the two previous years showed that grades 
and behavior ratings earned in the ARRIBA program were significantly above 
those of pupils before. the program. This result was not chosen for replication 
in 1971-1972 because of the high probability that the outcome would remain 
the same. 

Previous years' evaluations in the Model Schools have shown that 
Model A pupils master writing at the levels anticipated by project staff and 
that teachers rate pupils' classroom behavior in the normal ranges, but that 
teachers of each ethnic group report more "adaptive" classroom behavior in 
their rooms from pupils who share the language dominance of the teacher. The 
first year's evaluation also showed that pupils in the Bilingual Program had 
greater mastery of their first and second languages than had control pupils. 

Discussion and Conclusions 

Overall, this evaluation suggested that all three components of the 
program are conceptually sound. The major area needing attention appears to 
be program management, especially coordination of materials development, 
instruction, and supervision with one another and with project goals. 

The relative merits of Model A and Model D did not emerge clearlv. 
Rather, pupil performance varied in different areas and skills, with the 
variation as much attributable to difference in teachers and supervision as to 
the properties of the program itself. 

The evolution of the evaluation of the program from predominantly " 
criterion-referent, formative approaches to increasinglv summative approaches 
shows that, although the program has not met all the expectations of the persons 
who wrote its proposals, it has improved performances beyond those of base, 
line groups in most critical areas. 
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STUDY 1. GENERAL PROCESS 



EVAI.UATION OF THE THIP^ YEAR OF THE BILINGUAL 
PROG.RTU^ 



During the first two years of the Let's Be Amioos Program, 
process evaluation consisted of exaKiining, on a one-by-one basis, the 
successes of program management in implementing the enabling objectives 
and supportive procedures specified in the proposals of the project. 
Conclusions reached were that the. program was operated in a manner con- 
sistent with the objectives specified in the proposalis, with a few excep- 
tions: changes were deliberately made in the materials used for instruc- 
tion in second-grade classes; there were problems in implementing Model 
B programs; and curriculum development was slower than anticipated, 
especially in the ARRIBA program, preventing implementation of some planned 
evaluation . 

A number of changes have been made in the approach to process 
evaluation in 1971-1972. Some areas, such as curriculum development and 
use of new curricular materials in the ARRIBA program, have been handled 
in Separate studies because of their critical importance, as have questions 
of inportance to project management that were never written as process 
objectives or management goals. 

Other areas, not warranting special data-collection procedures 
and analyses, are discussed in this study. Rather than focusing on each 
enabling objective and supportive procedure, this third year's report 
focuses on the areas where discrepancies — either improvements over 
the original plans or instances where improvements were needed — existed. 

It should be noted that most monitoring data, results of the 
curriculum-development process, and detailed information regarding the 
population served were presented in the 1972-1973 Continuation Proposal. 

Procedures 

Program 

Model A . The Model A program continued to operate at the Potter- 
Thomas School site, where it encompassed prekindergarten throucfh third- 
grade classes, the last of these being new during the third operational 
year. • 

The Model A program plan was to team-teach, with Anglo teachers 
cind Latino teachers (nearly all of whom had at least a working knowledge 
of both languages) teaching pairs of classes. Pupils in each pair of 
classes met in linguistically mixed or homogeneous groups , with most 
instruction in homogeneous, mother-tongue groups in the earlier grades, 
and greater amounts of second-language contact in homogeneous and 
ethnically mixed groups in the later grades . 

During the typical school day, pupils moved betwieen teachers 
in the team, assuring that each child had some experiences in his mother 
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tongue and his second lanr-uaco, and : : ethnically honoaeneous and ethni*"^ 
cally diverse groups. Thr curriculum, specified materials to bemused, 
global pupil-performance coals to be obtained, and nicrbobjectives describ- 
ing tasks that pupils should have been able to perform. 

As the oupils progressed through the crad.es, their contact with 
the second languaoe was to increase from ter. percent of the school day 
at the beginning of the prekindergarten , ki::deraarten , and first grade. 
The pre kindergarten and regular kindergarten levels of the program met 
for a half-day of school; the reip.aindf r were full day programs. 

Beginning in second operatic nal year, a special, all-day kinder- 
garten program was developed, in which pupils beaan reading preprimer 
materials in their mother '-.ongue . This plan was maintained in 1971-1972, 
cind alumni of the 1970-1971 all-day kindergarten were enrolled in a 
special, enriched first-grade curriculum. 

Model B . This program sought to bring a bilingual school experience 
to smaller groups of English- and Spanish-speaking pupils within a school 
which was not reorganized as a bilingual program. At each site, at least 
two classes in one grade level — one class consisting of Spanish -dominant 
children and one consisting of English -dominant children — participated in 
the program. Classes received second-lanauage instruction (English as a 
Second Language or Spanish as a Second Lanauage) from itinerant special i^"^ 
in those subject areas. The classroom teachers had to be native speakers 
of the language in which they instructed the students. Anglo teachers of 
Anglo children did not need to be bilingual; Latino teachers of Latino 
children needed to have some use of Enalish; and teachers of both lish 
and Spanish as second languages needed to be bilingual. The two c ^ses 
were supposed to conduct joint activities where possible. 

The instructional objectives of the Model B program were identical 
witli those of the Model A program. 

ARRIBA . The ARRIBA program continued during the third operational year 
to offer grade- level-appropriate instruction using the Spanish language, 
in Science, Mathematics, Social Studies, and Spcinish as a First Language. 
English as a Second Language also was provided for students at all sites, 
as in previous years . 

Pupils in grades three to twelve were served, with the actual 
instructional format articulated to meet the demands of each instructional 
■level. In the elementary school the program was operated in self-contained 
classrooms. In the junior high school, where pupils move from class to 
class for each subject, students- were in Bilingual Program classes for 
part of the day and received instruction with Anglo classmates for the 
remainder. In the senior high school students had individual rosters, 
permitting the selection of courses from among those offered by the ARRIEA 
components . 

.V 

Evaluation 

This report is based on three sources: (a) monitoring data 



ERIC 



2 



gathered by procram supervisors: (b_; follow-up of these data throuch dis- 
cussions with the project director; and (c) information gathered informal.ly 
during numerous visits to schools by the evaluation staff. V^^ere data 
have been gathered to answer specific project-development problems, tJ-iey 
have been prepared as separate studies. 

Results and Discussion 

Model A . During 1971-1972, this proaram served 937 children in grades 
from prekindergarten to three. As was reported in the Continuation Pro- 
posal for 1972-1973, monitoring and classroom observation showed that the 
teaching procedures used in the proaram were consistent with the methods 
and processes described in the enabling objectives and supportive proce- 
dures described in the proposal. There were, however, important modifi- 
cations in the Model A program procedures which did not fit neatly into 
the objective framework. 

The first of these was an accommodation within the program to 
meet the needs of pupils in the second and third grades transferred to 
the Potter-Thomas School without prior Bilingual Prot^ram experience. In 
the past, such children would have been assigned to a nonprogram class at 
the pupils* grade level at the school site. As the number of transfer 
pupils grew, it became clear that this could not be continued. As a 
result, a mixed-grade pool of pupils was formed in a large room accommo- 
dating four classes and a smaller adjacent room, to be taught by three 
Latino and two Anglo teachers. The classes taught by these five teachers 
were three third-grade classes, one second-grade class, and one mixed 
second^/third-grade class. Like "the. instructional pattern in the early 
grades and the other second grades, pupils were divided into linguisti- 

'cally homogeneous groups and moved among the teachers. But, in contrast 
to that pattern, children were grouped, not on the basis. of their grade 
level, but on the teachers* judgments of pupil competence in the various 

■curricular areas. This permitted the accommodation of pupils with varying 
amounts of experience in Spanish (whether as a first language or a second 
language) in groups of pupils with similar levels of skill. All of the ■ 
competency -base.d instructional groups covered a similar range of materials 
so that it remained possible to evaluate the pupils according to their 
grade level as stipulated in the objectives and microob jecti ves . hs a 
result, major revisions of the program objectives were not required by 
the changed organization of instructional groups. 

The second area in which change occurred was the teaching of 
reading. During the second operational year, reading instruction of 
.Anglo pupils was diffuse, with phonics-based instructional materials 
introduced in some Anglo classes and the pro ject-staff -selected Bank 
Street text introduced in others. During the 1971-1972 school year, a 
uniform procedure was reinstated for reading instruction in the Anglo 
group. Except for the all-day-kindergarten , where the Chandler Pre- 
primer reading sei'ies was Used, the main reading text was the Bank Street 
series, with. the Lippincott readers used as a supplement. The use of 
Lippincott as a supplement had not been specified by the program; however, 
the Bank Street series provided reading material for use only to the end 
of the third grade. Because a decision had been made to wse Lippincott 
readers in the fourth grade in 1972-1973, when the prooram would reach 
that grade level, the use of Lippincott as a supplement assured a smooth 
transition from one reading series to the other. 

ER?C 



The third area of rodif ication of Model A processes was staff 
development. Tho procrarr. proposals specified that staff development would 
consist of progrars and discussions with a variety of experts in the 
bilingual field, Hispano-7\merican culture, and the curricular areas rele- 
vant to the Bilingual Program. During the course of the 1971-1972 school 
year only one svch speaker was presented, and that at a workshop for school 
system people includina those not a part of the Bilingual Program. Accord- 
ing to supervisors, staff development during the year consisted primarily 
of meetings held to address specific problems: curriculum development 
and planning; coordination of team roles; development activities; and 
problems observed during the monitoring process. These meetings were 
held in alternate weeks throughout the school year. In addition, super- 
visors reported that separate Bilingual Program staff -development time 
was provided during most faculty meetings. The program suoervisors were 
responsible for planning and chairing these supervisory m.eetings. 

Model B. During 1971-1972, the Model B component served 277 child- 
ren in grades one and two. Although similar to the plan described in the 
original 1970-1971 Continuation Proposal, it still encompassed a variety 
of teaching strategies and structures designed to permit the fitting of 
bilingual education into existing school structures. 

At the McKinley site, it was iirpossible for project staff to 
come to agreement with the principal regarding the nature of bilingual 
education. The staff felt that Spanish, when it was the pupils* mother 
tongue, should be used Tor most of the instructional day in the early 
phases of the program; the principal felt that English and English as a 
Second Language ought to predominate. In October it was decided by mutual 
agreement of the principal and. project staff to withdraw the program from 
that school . 

i 

At other sires, the structure of the program was modified to 
accoiranodate local needs and resource limitations. At Bethune School, one 
team consisting of a first-grade Anglo class and a first-grade Latino 
class was operated. Each was taught in all s^ibject areas other than 
Second Language by a teacher v^ho v/^s a native speaker of the pupils' 
mother tongue. A Second Language specialist taught the Latino group 
English as a Second Language and the Anglo group Spanish as a Second 
Lariguago. (During the remainder of the day, the specialist taught ESL 
to other children at the school.) Latino and Anglo classes reportedly 
held joint activities for half--hour sei sions twice a week, but no record 
of observation of these activities appeared in the monitoring reports. 

At Miller School, one class consisting of ll.atino<^, mixed first 
■ and second graders, was operated. This was a self -containc i class taught 
by a Latino teacher. An FSL specialist assigned to the school worked with 
the teacher providing English instruction for half the school day. This 
ESL specialist often worked by drawing out small groups of children with 
similar levels of skills. 

At Ludlow the Model B program operated in four classes, Anglo 
and Latino first-grade groups and Anglo and Latino second-grade groups. 
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Each class was taught by a native speaker of the pupils* mother tongue. 
A Second Language specialist provided instruction in ESL and SSL for 
these classes. According to supervisors, the Anglo and I.^tino groups held 
joint activities "once in a v;hile, not regularly." Exaniination of the 
monitoring forms shows that no joint activities were observed. 

In the course of data gathering for other parts of this report, 
it was noted that the Anglo-Latino distinction does not apply consistently 
to the Ludlow class groups. Elsewhere in the Model School programs, the 
children were assigned to language aroups on the basis of their lan- 
guage. Thus, pupils who qrew up in households w^iere Spanish was the main 
language were designated as Spanish-speakina (Spanish-surnamed children 
whose parents consistently used English in the home were, of course, 
treated as Anglos for most instructional purposes.) 

In. the Ludlow Model B program, however, it was found that the 
so-called Anglo groups contained children who came from Soanish-speaking 
families but knew enough English to get by in classes where English was 
used. This pattern is likely- to have made Model B "Anglos" appear 
better in Spanish and poorer in English than a true Anglo group. 

It should also be noted that at those schools where Anglo groups 
were operational, the instruction of these groups in English was not 
imder direct project control. Rather, the English-language teacher pro- 
vided instruction (in all areas but SSL) which was consistent with practices 
elsewhere in the school, but not with those o^ the Bilingual Program. 

In addition to modifications in the organizational structure 
of the classes, the Model B program began serving as an experimental 
center for the tryotit of new materials produced by the Miami Curriculum 
Development Center. As tryout centers, first-grade classes used materials 
in five areas: Language Arts (Spanish as a First Language) , Spanish as 
a Second Language, Fine Arts, Social Studies, and Science/Math. First 
copies of these materials appear to have a substantial overlap with stated 
objectives and mixed objectives of the Bilingual Program. Therefore, no 
revisions in the objectives were made. Hard data describing teacher 
reactions to the materials and some assessments of student performance 
were collected by the Miami Curriculum Development Center. Results were 
to be, but have not yet been, shared with the project evaluator. Informal 
discussion with teachers and supervisors suggested that they felt the 
materials were satisfactory but too brief to constitute the major course 
of study in the areas written. into them. Instead, they have been used in 
the main as a supplement to teacher-planned activities. 

Staff development in the Model B program was similar to that 
of Model A, in that it focused on, problems at each school site, discovered 
in the supervision process or brought to the supervisors by the teachers. 

AR RIBA . The major change in this program was an expansion of the 
services provided to high school students. Dxiring the first operational 
year, ARRIBA served pupils in two elementary schools (Ludlow and Waring) , 
two junior high jschools (Stoddart-Fleisher and Penn Treaty) , and two 
senior high schools (Edison and Kensington) . In the second operational . 
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year two senior high schools, 'villian Penn and Benjajnin Franklin, were 
added to the prcaram, with one pair of teachers sharina tir?ie between the 
schools. 

During 1971-1972,' the teaching force at these two high schools 
was expanded, so that each now has two bilingual faculty inembers . This 
school year also saw the beginning of development of a program of secre- 
tarial studies at Kensington High School. The courses offered were 
Clerical Practice and Typing. These classes were conducted in Spanish 
and aimed at producing skills in Spanish. (The students could also, 
elect courses teaching these skills in English) . According to the 
supervisor of the program, the typing course was conducted entirely in 
Spanish but the Clerical Practice course of study included use of a 
text in the Encflish language . 

At one senior high school (Penn) , there was a disagreement with 
the principal regarding the amount of English contact students had. A 
Compromise was worked out in which students who had good facility in the 
English language were encouraged to take more subjects in that language. 

Staff development for the ARRIBA program was more formal than 
it had been in past years. According to information provided by the 
program supervisor, staff meetings were held on alternate weeks. Checks 
with the Curriculum Development Coordinator's office showed that they 
were held less often. According to the Coordinator's record there were 
only five during the year, (in October, November, December, January, cind 
April) . Major activities during these staff meetings followed two themes:-^ 
(a) development of awareness of the problems of Puerto Ricans and other 
Spanish -speaking in the mainland schools, and (b) exploration of curri- 
culum coordination skills. .According to information provided by program 
supervisors, films were shovm , materials were examined, and discussion 
groups were held. Videotapes of teacherjs in their classrooms were made 
cind discussed in order to upgrade teaching skills. Dr. Krogh , project 
coordinator for curriculum development, coordinated most of these meet- 
ings . 

Other Program Processes . A workshop in Puerto Rican History and 
Culture was held jointly with the Social Studies Department of the School 
District (Dr. French, Director) in May, 1972, in which high school teachers 
from all across the city participated. The program included an overview 
of the history of Puerto Rico, presented by Dr. Carrion, Professor of 
History at the University of Puerto Rico and former Undersecretary of 
State for Latin 7\inerican Affairs. • The program was sponsored by the B*nai 
Brith Anti-Defamation League, with whom project staff are developing high 
school social studies materials about Puerto Ricans. About 40 teachers 
attended this conference. 

A Summer Teacher-Training Institute was he.ld in 1971 to train 
■ Spanish-speaking persons as teachers, and provide them with emergency 
certificates. Twenty teachers, all native speakers of Spanish or from 
Spanish-speaking families, were enrolled and completed the proaram. All 
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but two taught in the Philadelphia schools during the 1971-1972 school 
year". 



This study suggests that the major modifications in the Bilin- 
gual Program have come about largely in response to chances in the environ- 
ment in which the program operates. Those of \Model A were due to shifts 
in the population of pupils being served , those of Model B were due to 
the need to fit the program into existing school structures, and those. of 
ARRIBA were in response to "student interest in commercial siobjects and 
an increased demand for the program. Changes in these respects are tp 
be anticipated as parts of the evolution of the program. 

Admittedly, the program management has to focus on some complex 
problers which affect fundamental program processes. However, those 
problems were explored separately from this process evaluation and can 
be found in Studies 2 and 3. 



Conclusions 
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study 2. PERCEPTIONS OF PRINCIPALS, TEACHERS/ Ar^JD PARENT.? REGARDING 

THE BILINGUAL PROGRAM 



During the first year of the Let's Be Amigos project's ooeration/ 
principals were surveyed in order to assess their evaluations of program 
components operating in their schools. In the second operational vear, 
surveys of parents and pupils served by the project were added to the survey 
of the principals. 

For the third operational year, it was felt that two of the surveys 
should be replicated. The nrincipal ^s survey needed replication because it 
assessed current status of the project. The parents' survey was replicated 
because the previous survey was not anonymous/ i.e*, parents were asked to 
indicate the name of their child in the program and questionnaires were 
returned to classroom teachers. It was felt that these features might bias • 
results in favor of the program. 

Overall results last year were generally complimentary to the 
program: most principals were satisfied with program operation, the only con- 
sistent problem being in the area of supervision of teachers; nearly all 
parent and most pupil reactions . indicated a high degree of satisfaction. 
The principals' reports that there were problems in supervision suggested that 
teachers be polled in order to find out the kinds of supervision they were 
receiving. A questionnaire was then added for teachers to indicate^ their own 
feelings about supervision. The teacher questionnaire also provided opportunity 
for the project administration to gather data about curriculum materials being 
used and curriculum materials that teachers wanted but did not have. 

Procedures 

Principals ' Survey 

This survey was a replication of the . data-gathering process used 
during the second year of operation. 

Instrument . The principals' questionnaire appeared on page 219 of ■ 
the Second Year, Evaluation Report (Offenberg, 1972). It asked principals to 
give- their overall impressions of the program and asked for specific information 
about pupil, parent/ and teacher reactions to the program/ about supervision/ 
and about the" performance of teachers in the program. The instrument provided 
opportunity for the responding principal to qualify or comment on the ratings 
he gave. 

Method . A copy of the instrument was mailed in March to principals 
in whose school a program component was operational. Telephone follow-up 
assured that each principal returned a questionnaire. All were mailed to the 
project evaluator. The cover letter assured that individual respondents would 
not be identified in any reports or to other project personnel. 
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Subjects . All principals whose schools were served by the project 
were included in the study. 



Analysis. The principal's ratings were tabulated and percentages 
computed. Responses to open-ended questions were tabulated. 

Teachers' Survey " 

Instrument . The teachers* questionnaire/ shown in Appendix 2,1, had 
two parts. The first part was designed to survey the materials teachers were 
using and ^the materials that teachers wished to have. This part of the 
questionnaire required that the teacher be identified. The second part of 
the questionnaire, which was designed to assess the strengths and weakness 
in the supervision process, was anonymous - 

Method . All teachers who attended a staff meeting in February, 1972, 
were given- copies of the questionnaire to complete. Teachers who were not in 
attendance were mailed questionnaires. The questionnaires were accompanied by 
envelopes which the respondents could use to return separately the signed and 
the anonymous questionnaire sections. 

Subjects . All teachers working in any of the three instructional 
components of the program were surveyed. 

Analysis . Results were tabulated and percentages computed where 
"appropriate. 

Parents' -Survey > 

This survey, was a modification of one conducted in 1970-1971, the 
second year l^f program 'operation. The content of the survey was similar to 
that of the previous survey, but the data-collection procedure was modified 
to increase the . anonymity of 'the respondents. 



Instruments . The questionnaire was prepared in two versions — one 
English, one Spanish. They are shown in Appendix 2,2. The items in both are 
nearly identical, the variations being permitted to assure that both versions, 
included idiomatic statements in their respective languages. The questionnaire 
was designed to tap in a very simple way the parents* perceptions of the program, 
whether they had contact with the project through parent activities. The 
modification in the questionnaire was a revision of two items which not only 
asked parents to indicate their knowledge of the program aims, but also permitted 
them to indicate their satisfaction with their children's progress toward 
those aims . 

Method . Each teacher in the program was supplied with enough 
questionnaires, cover letters, and stamped return envelopes for his pupils. 
During the third week of March, .follow-up showed that all teachers had distrib- 
uted the materials to their classes and provided appropriate instructions: 
parents were to complete the questionnaire, using the language of their pre- 
ference, and mail it to the program-evaluation staff. The cover letter and the • 
teachers both made it clear that parents and pupils were not to identify them- 
selves. 
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Subjects . Teachers were asked to see that all pupils who were 
in attendance during the two or three days following the distribution of 
the questionnaires received them, however, they were not asked to keep a 
specific record. It was not therefore not possible to know exactly how "many 
pupils received questionnaires. The percentage of questionnaires returned, 
based on the number of pupils on roll in the program, was computed and 
appears in the Results section of this report. 

Analysis . Results were tabulated by program and school level 
(elementary, junior high, senior high school) . Numbers and percentages of 
each of the responses to the questions were tabulated, as was the percentage of 
the pupil population for whom questionnaires were returned. 

Results . 

Principals ' Survey 

All 11 principals of schools with bilingual programs funded through 
Title VII .responded to' the questionnaire. 

The first question on this instrument asked principals to indicate 
their overall level of satisfaction with the bilingual program components 
operating in their schools. Two principals reported they were "very satisfied," 
eight stated that they were "somewhat satisfied," and one stated that he was 
"somewhat dissatisfied." No one was "very dissatisfied." Of the two principals 
who were "very satisfied," one, the principal of an ARRIBA junior high school, 
said that there was need for more interaction of program teachers with non- 
Spanish -speaking children. Five of the eight "somewhat satisfied" principals 
commented. Two (one of a Model B school and one of an ARRIBA program school) 
felt that more supervision was needed from the central staff. Two others (one 
of a Model B school and one of an ARRIBA school) felt that the program for 
English-speaking pupils needed improvement. Another principal felt a need for 
more emphasis on speaking and reading English (ARRIBA) . The one principal 
who was "somewhat dissatisfied" focused on curriculum materials. He felt that 
there were inadequate materials in social studies, Spanish as a First Language, 
and mathematics. He also said that English-as-a-Second-Language materials, 
while good, were in insufficient quantities (ARRIBA) . 

• Question 2 asked principals about expanding the program in their 
schools. Seven of the 11 respondents wanted expansion, three felt the program 
in their school should be kept the same size, and one felt the scope of the 
program should be reduced. No one indicated that the program should be stopped 
in their school. Three of the principals requesting expansion of the program 
commented on their answers. Two cited the increasing size of the Spanish- 
speaking population in their schools (Model B and ARRIBA). .One also noted 
that an on-site supervisor was desirable (ARRIBA) . One principal indicated that 
addition of another bilingual teacher to his staff in order to provide smaller 
classes for "intensive assistance in language development" would be disirable 
(ARRIBA) . The principal who wished to have the program reduced stated that he 
felt the classes for English-speaking pupils should be dropped, but the program 
for Spanish-speaking should be expanded (Model B) . . 
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Question 3 asked principals who wished to have the program expanded 
whether more teachers with the background and the training of the summer 
institutes would be good additions to the staffs (see appendix 2.3 for infor- 
mation about summer institute) . All seven principals desiring expansion answered 
affirmatively. The principal who asked that the program be reduced through the 
dropping of the English language component of Model B also indicated he would 
like a:dditional institute-trained Spanish-speaking personnel. Two principals 
commented on their answers (other than to indicate how many teachers would 
be required) . One stated that he wished the teachers had better fluency in 
English, and one said that he wished the teachers were more oriented to the 
local school system, classroom management,, instructional techniques, school 
organization, and curriculum-development procedures. 

Question 4 asked principals to rate the supervision received by 
teachers at the schools. One high school principal failed to answer the 
question, stating that the terminology needed clarification- One principal 
thought it was excellent (Model A) ; four (three ARRIBA and one Model B) thought 
it was good; and five thought it was fair (three schools having ARRIBA components 
and' one school with both ARRIBA and Model B) . None felt that it was poor. 
Four principals, all rating the supervision as "fair," wrote comments. Two 
felt that the amount of supervision was insufficient, one principal asking for 
concentrated blocks of time rather than "the hello-and-gootibye technique" 
(Model B) ; the other pointed to the great need for assistance of most of his 
teachers (ARRIBA). One principal (ARRIBA) said that supervision needed to be 
more structured and content oriented. One principal (ARRIBA) indicated that 
his school needed an on-site supervisor recognized by students as part of the 
regular school administration. 

Question 5 asked principals to suggest areas where the summer 
institute's training of teachers could be improved. Two principals omitted 
this question. Of the remainder, six made suggestions. Two emphasized 
bilingualism, 'Stating that the training ought to produce a greater commitment 
to all children and provide equal emphasis for English and Spanish. Develop- 
ment of activities and materials for students, teaching of reading, involvement 
of principals as participants and group teachers, and managing classrooms were 
emphasi-zed by "the other principals. 

Question 5 asked principals to indicate whether there was increased 
understanding of the program goals this year (as compared to last year) by 
oarents, students, and faculty. Seven of the nine principals who responded 
about parent understanding indicated that there had been gains. Nine of eleven 
respondents felt there had been gains in students' understanding. Ten of eleven 
respondents felt there had been gains on the part of faculty. 



One principal (ARRIBA) commented that there had been an increase in 
the number of parents coming to the school. Another (ARRIBA) who indicated 
that there had been gains for all groups felt that more dissemination of 
information was still possible. A third principal felt that goals of the pro- 
gram should be distributed at the beginning of the year, and that the evaluative 
program should be designed in relationship to these goals. A fourth principal 
who felt that there were no gains for any groups wrote, "Some of the teachers 
have no idea of the goals and objectives of the program. Also, they are unsure 
of the timetable for the introduction of Spanish and English." 
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Question 7 asked principals to note other factors about the program 
which they wished to bring to the evaluator's attention. Three principals 
asked for greater articulation with the school staff and/or administration. 
One principal felt that a teacher at his school was anxious to put the Bilingual 
Program into an autonomous department which was contrary to school . philosophy. 
One principal indicated that there was a need for a diversity of curriculum 
materials. The last principal indicated that the teachers were cooperative, 
mature, and dependable, but they needed appropriate textbooks. 

The last section of the principals' questionnaire called for ratings 
of staff. The results overall showed that principals ' views of teacher per- 
formance were favorable. As can be seen in Table 2.1, 91% of the ratings were 
excellent or good. The three -areas where principals saw the most teachers 
as being fair or poor were relations with fellow teachers, preparation and 
organization of class materials, and class control. Even in those categories, 
the number of teachers rated fair or poor never exceeded 14% of the teaching 
staff. 



Table 2.1 Siimmary of Principals* Rating of Teacher Performance, All 
Teaqh.^?r.S.!.,,. Groups and Components Combined. 





Item 


Excellent 


^:Good 


Fair 

< 


Poor 






N 


% 


N 


% 


N 


% . 


N % 


1. 


-■ . 

Rapport with students in 
his/her class. 


36 


58 . 


22 


35_ 


1 


2^ 


3 5 


2. 


Relationship with 
administrative personnel. 


34 




26 


42^ 


1 




i 

r 

! 

1 2_ 


3. 


Knowledge of the subject 
taught.. . " 


26 


41 


36 


56. 


1 


2 


1 2 


4. 


Ability to relate to 
fellow teachers. 


24 


is 


31 


48 


: 8 


hi 


1 1 2 


5. 


Preparation...^nd 
organization of'- teaching 
materials 'and lessons. 


30 


£7 


: 27 


42 


1 ■ 

I ^ 


9 


i 

i 1 2_ 




Class control. 


28 


45 


26 


42 


4 


6_ 


j 4 6 


^otal (N - 378) 


178 


47 


! 168 


44 


21 

t 


6^ 


! 11 3 
i - 
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Teachers ^ Survey 



Part I of the teachers' questionnaire (which ■ focused on supervision) 
v/as returned by 53 of the 63 teachers in the program (87'*.) . The questionnaire 
asked teachers to provide information on their backgrounds and job roles as 
well as their perceptions of the supervisory process. Results are shown in 
Tables 2.2 through 2.4. Results for the first two questions (Table 2.2 and 
2.3) were Classified by the-:- ethnic background of teachers. Among Spanish- 
speaking teachers, a substantial number of whom were teaching in the program 
for the first time during 1971-1972, responses were further divided into 
teachers^ who were new, and teachers who had at least one year's prior experience. 

Results overall were similar for all three groups. Roughly half 
of the respondents reported that supervisors helped them with materials. 
About a quarter of the teachers stated that they received help in other school- 
related areas, especially teaching methods. Spanish-speaking teachers reported 
receiving this kind of help more than twice as often as English-speaking 
teachers. Slightly less than one quarter of the teachers responded that help 
was given providing information or orientation about the school or the program. 

Attention must also be drawn to the 15% of the respondents (21% of 
them in th« "New Spanish-speaking"- category) who felt that little or no help 
had been received from supervisors. This result suggested that at least 
some teachers, especially inexperienced ones, may need additional attentiori 
from the supervisory staff. 

The kinds of help desired, appearing in Table 2,3, parallel/ to 
some extent, the kinds of help that the teachers reported the supervisors 
gave. The first four categories all focus on the distribution and use -of 
teaching materials, with a general request for information about theu appearing 
on 23% of the returned questionnaires (and on 47% of those of the new Spanish- 
speaking teachers) . More specific curriculum-material requests appeared on 
from 2% to 9% of the remaining questionnaires. (The percentages cannot be 
added because some teachers appeared in more than one of the four categories). 
Other types of help discussed appeared relatively infrequently. They are 
shown on the remainder of the table. The last line of the table is a count 
of the number of teachers who wrote "None" or left the item blank, presumably 
indicating that there were no kinds of help they wanted but were not receiving. 
As can be seen, roughly one-third of the teachers (36%) were in this category, 
but the. three respondent groups varied markedly: only half as many new spanish- 
speaking teachers left the item blank as had English-speaking and experienced 
Spanish-speaking teachers. 

Only a sinall number ("15) of the teachers used the last open-ended 
item to bring salient facts to the attention of the project administration. 
Their comments, organized by program component in Table 2.4, speak for themselves. 

The second part of the teachers! survey asked teachers for the materials 
they had been using and the materials they wanted. This was designed primarily 
for project management, and was reported to the project director in March, 
1972. The results.lwere summarized in the 1972-1973 Continuation Proposal for 
the project. Since the survey,, other materials were distributed and used by 
t^e project staff. As a result, the following count has been updated to 
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Table 2.2 Teachers' Responses to the Item, "List the ways the super- 
visors in the program have been helpful. to you/' as a function 
of teacher ethnic group and experience in the program. 







Teachers Responding 




Type of Help 


Eng 


. -Speak 


, Tchr.^ 


Span . -Speak , 


Tchr, ^ 


Total 




Received 


New 


St Experienced 
N 


New & Experienced 
% ' ' 'TT • • % 


N % 


1. 


Provided or helped 
with books, materials, 
etc . 








40 


10 53_ 


10 


53 


26 49_ • 


2. 


Provided personal 
or teaching methods 
in school related 
area . 






2 


13_ 


6 31 


6 


31 


r 

:.4 26_ - 


3. 


Provided orientation 
and information. 






3 


20 


3 . 16_ 




31 


12 22_ 


4. 


Provided information 
about school rules, 
discipline. 






0 


0 


2 11^ 


1 


5_ 


3 6 


5. 


Miscellaneous help 
provided. 






1 


r 


1 5 


1 




3 6 


6. 


'*None," "in no way," 
or critical of help 
received . 






2 


13_ 


2 11^ 


4' 


Ik 


8 rs 


7. 


No response to 
question. 






2 


13_ 


4 21^ 


1 


5 


7 12 



^As only two responding English speaking teachers were without prior 
experience in the program, results for all English-speaking teachers are 
combined. All English-speaking teachers are part of Model A or B components. 



2 

New teachers worked in the program during the year 1971--1972 for the 
first time. Virtually all new teachers had no prior experience in the 
Philadelphia schools. 
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Table 2.3 Teachers* Responses to the Item, "List the kinds of help you 



would like to receive from the supervisor of the program." 





Teachers Responding 


Type of 


English-Speaking 


Spanish-Speaking 


Total 


Help 


New § Experienced 


New & Experienced 





Desired ; (N=15) j (N=19) {N=19) i N=:53 





N • 


% 


N 


% 


N 


% I 


N 




Information about \ 
materials . ; 


1 


7 


2 


{ 

11 i 


9 


47 


12 


23 


What material is 
avaixaD±e. 


A 

4 


zb 




j 

1 

} 


u 


u 


/ 




How to use materials . 


2 


13 


3 


16 


0 


0 


5 


9 


Follow-up on requisition 
for materials . 


1 


7 


0 


0 


0 


0 


1 


2 


Provide copies of objectives 
early in the year. 


3 


20 


5 


26 


0 




8 


15 


Improve supervision, 

uJ.<^ j.oin, luc/jTc iux,i.ow— up . 


U 


U 








±o 


c 
D 




Revise curriculum. 


0 


0 


1 


5 


2 


11 


3 


. 6 


Develop second-language 
instruction 


3 


20 


0 


0 




5 


4 


8 


Improve staff 
development. 


1 


7 


i 2 

1 


11 


1 


5 


4 


8 


Help with community 
activities . 


0 


0 


\ 

\ 

! 1 


5* 


1 

1 


5 


2 


4 


Miscellaneous. 


3 


20 


1 2 


11 


6 


32 


11 


20 


■ 

None or no response. 


7 


47 


3 


42 


4 


21 


19 


36 
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Table 2.4 Responses to the Questionnaire Item, "Use the space below to 
bring to the attention of the project staff anytning else you 
wish." 



Model A 

would like opportunity to offer some ideas about the program and how 
I think we can improve it in some ways. 

Would like permission to observe classes locally and outside of 
Philadelphia. 

Would like to secure buses and tickets for student trips to events,', 
museums, etc. 

Model B 

. ' Would like special help for some students in classroom. 

There are too many surveys, and not enough solutions to problems. 

Would like more curriculum materials relevant to Puerto Ricans on 
the mainland, not just material about Puerto Rico's culture. 

ARRIBA 

Would like workbooks and exercises for books now in., use (in the junior 
high school level) . 

Would like more uniformity of materials in similar courses at various 
sites in the program (at the high school level) . 

Coordinate rostering (at the junior high school level) to benefit 
pupils who need only hcilf of their classes in Bilingual Program. 

Make appropriate classrooms for specialties (science ,. etc . ) available 
to bilingual teachers at the high school level. 

Hold discussion groups of teachers throughout the program who teach 
the same subject areas. 

Develop a special student center, with information on Puerto Rico, 
a library, etc . 

Arrange demonstration classes with publishers to find out about new 
materials (like the workshop last spring for science kits) . 

Third- and fourth-grade ARRIBA teachers would like to have an established 
curriculum for their grade level, and would work without pay to develop 
it. 

* ■ 

. ■ would like closer corl'tact (of projec.t. staff) with teachers. 
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include not only the information appearing on the questionnaires, but also 
information gathered in visiting classrooms. 

The latest findings (May, 1972) show that 43. materials were 
completed and readied for distribution; of these, sixteen (39%) were 
actually in use. As a result of these findings, a separate assessment of 
curriculum distribution in the ARRIBA component was prepared. It appears 
as part of Study 3. 

Parents' Survey 

As of May^ 1972, there were 1,813 students served by the bilingual ■ 
program. It was estimated that when the parent questionnaires were sent 
out in March, there were about 1,700 pupils being served by the program. Of • 
these, 538 (about 3 2%) returned questionnaires to the evaluation staff. As 
shown in Table 2,5, the amount of return varied by grade level with the lowest 
rate of returns from the high schools (14%) . Among those parents who indicated 
the school and grade (permitting program identification) were approximately 
29% of the Model A parents, 36% of the Model B parents, and 20% of the ARRIBA 
parents. 

Table 2.5, also shows language-usage patterns. Results from Models 
A and B showed that, although about equal numbers of English- and Spanish- 
speaking pupils were enrolled in the components, Spanish-speaking pr rents 
were more likely to respond than were the English-speaking. As anticipated, 
virtually all ARRIBA program parents reported that they were Spanish-speaking 
.or bilingual . 

Questions 4 through 8 on the table assessed parent perceptions of 
the. program in terms of their having basic information and their having favor- 
able or unfavorable reactions to its operation • Results were very uniform 
for all the components of the project- Most parents responding (89% program 
wide) were aware that English was being taught in the program, and most (93%) 
were aware that Spanish was being studied in the program. Of those respondents 
who were aware of the use of each language, over 90% felt that their children 
were making satisfactory progress in the languages, and reported that they 
liked their children to study two languages and wished to have them continue 
to do so. 

■ Questions 9 through 11 asked parents whether they had had contact 
with project teachers or other project personnel. Results showed that about 
half the responding parents in each component had contact with their child's 
English-speaking teacher, Spanish-speaking teacher, and/or the project staff ^ 
through a meeting at the school. ' 

Table 2.6. shows the responses of Model A and B parents who are 
English-speaking (who said they speak to their child only in English on 
Question .1) or Spanish-speaking (who reported that they speak to" their child 
in Spanish and English ). Spanish-speaking parents were more likely to respond 
than were English-speaking parents. The 160 Spanish-speaking Model A respondents 



ERLC 



23 



w-3re about 32% of the group; the 101 English-speaking Xodel A parents were 
about 24% of the group; the 74 Spanish-speaking jModel B parents were 50% 
of the Model B group, the 17 English-speaking parents who responded were 
about 15% of those involved in that program. 

. Results for the parents who responded to- the questionnaires in the 
two ethnic groups were quite similar in their liking of program. Over 90% 
liked their child to be learning two languages, and luost (at least 80%) of 
each responding group wanted their children to continue doing so. There 
were, some interesting differences in other areas between the two ethnic 
groups. While the great majority of Model A and iModel B Latino parents were 
aware of the study of both English and Spanish in school, a surprising number 
of Anglo parents seemed confused. Seventeen percent of the responding English- 
speaking parents of Model A, and 20% of the (admittedly small) group of 
English-speaking parents of Model B, seemed to think their children were not 
studying English. In addition, 13% of the Anglo parents of Model A seemed 
unaware that their children were studying Spanish, A second interesting point 
is the relationship between the ethnic group of parents and the ethnic group 
of the teachers that parents have visited. While' a majority of both Spanish- 
speaking and English-speaking parents reported visiting their child ' s ' English- 
speaking teacher, the English speaking parents said they visited the English- 
speaking teacher twice as often as they visited the Spanish-speaking teacher. 
There were no clear-cut differences between the groups regarding the frequency 
with which Spanish- and English-speaking parents visited the school for other 
meetings . 

In addition to. the choice-response items, parents were invited to . 
make comments regarding the effectiveness of the program. Programwide comments 
are shown in Table 2.7, As can be seen/ the Latino parents made comments 
twice as often as Anglo parents. Overall, the comments were highly complimentary 
to the program, with the number with expressions of agreement with the program 
exceeding all other categories combined. It should be noted, however, that 
among those Anglos who commented, about one in ten expressed disagreement with 
the program procedures or goals, but among Latino parents less than one in 50 
expressed such disagreement. 



Discussion 

The outcomes of this assessment of principals', teachers', and parents' 
views of the program largely confirm the findings of last yearns studies, that 
there is general satisfaction with the program, but room for improvement. 
School personnel, especially principals, continue to feel that improvement 
is needed in supervision and in areas tied to courses of study (statement of 
program objectives and materials distribution) . 

The number of returns on the parent questionnaire was approximately 
that predicted by the School District's Office of Adminsitrative and Survey 
Research for a mail questionnaire without any follow-up (the method used in • 
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Table 2.1 Parents' Responses to the Item, "Please write any suggestions 
you have for the bilingual program." 





Percentage of 


Responses 


Response 


Anglo 
(N=118) 


Latino 
(N=420) 


Express agreement with or 
praise of program. 


21% 


51% 


Express disagreement with 
program. 


3% 


1% 


Suggest minor improvements 

J.il UlC ^ i- \J\^ i- GlVW V C . <^ . , 111(^1. c 

homework, more conversational 
Spanish , etc) , 


5% 


11% 


Request more bilingual 
personnel, administrators, etc. 


0% 


3%. 


Miscellaneous reactions. 


4% 


1% 


No response. 


67% 


34% 


Total 


100% 


100% 



the study) . The relatively low rate of return of questionnaires from the 
high school level was also predicted by them because high school students 
are less likely than younger pupils to take the questionnaires home. This 
does not necessarily indicate limited parent interest. 

The rate of return of questionnaires and the generally favorable 
reactions to the program recorded on them suggest that most parents are 
pleased with their children's experience with the project and do not (even., 
with the great effort made to collect anonymous data) bring up critical 
points . , . 

The results of questions about language usage in the home shows 
that the program components did^ in fact, reach their target groups. 
Model A and B reached children of both Spanish and English speaking 
households, the ARRIBA program reached children of Spanish-speaking house- 
holds. 
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The marked difference in nhe frequency with which Anglo and 
Latino parents reported having visited the teachers of the two ethnic 
backgrounds suggests that, while Spanish-speaking parents feel that both 
Spanish and English are important, the Anglo parents may either (a) feel 
very uncomfortable with the Spanish teachers or (b) feel that their child's 
study of Spanish is less important than his' study of English. The relatively 
low return rate of questionnaires from the Anglo parents probably indicates 
that the second of these alternatives is the correct one. 

The rather surprising fact that some parents / especially Anglos 
in Model A, think their child is not studying English, suggests that the 
effort to do the important job of communicating the goals of the program 
to parents might need improvement in the coining year. 

Results of the survey of teachers showing that the project-developed 
curriculum materials were being used only partically is also cause for con- 
cern. Discussion with the project director when this finding became available 
has led to some follow-up (see Study 3) which will be useful in remediating 
the situation next year. 
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X 

Appendix 2.1 Instrument Used To Survey 
The Bilingual Program Teachers 
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SCHOOL DISTRICT OF PHILADELPHIA 

Teachers Survey 
Title VII Bilingual Program 
Let's Be Amigos 

PART I 

Date_ . 

1. Check the program in which you are teaching: 

C3 Elementary School Model A (Potter Thonwis) 

D riementary School Model B (Miller, Ludlow, Bethune) 1st and 2nd grade 
□ Elementary School ARRIBA (Ludlow, Waring) 3, 4, 5 and 6 grade 

Junior High School ARRIBA (Stoddart, Peim Treaty) 
Q Senior High School, ARRIEA (Edison, Kensington, Franklin, Penn) 



2. • Check the one which applies to yo\ir teaching assignment - 
Q I teach primarily in Spanish. 
C3 ^ teach primarily in English . 
>CJ I teach both in Engj.ish and Spaziish , 



3. Check the one which applies to you. 

My mother tongue is English . 
Q My mother tongue is Spanish . 

4. Check hereQif this is your first year teaching in the bilingual programs. 

5. Please list the ways in which supervisors of the program have been helpful 
to you._^^ ^ 



6. Please list the additional kinds of help you would like to. receive from the 
supervisors of the 'program 



7. Use the space below to bring to the attention of the project staff anything 
else you wish.^ 
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PART 2 

Teacher Name Grade .s) Taucht. 

School 

!• In coliiion A list each subject 
you teach. 



A . ■ . B 



2. List by title and author any instructional materials which would be useful to 
you but which you do not now have , 



In column B write down the 
materials , text-books , etc , 
which you are using, or plan 
to use for each subject you 
wrote in column A. Put the 
texts on the line beside the 
subject for which it is used. 
If you do not use any materials 
in a subject, write "None". 



3, Are there any special, projects in which you or your pupils are involved? 
Please list them« 



Prepared by the Office of Research 

and Evaluation 
January, 1972 
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Appendix 2,2 Questionnaire Sent To Parents of 
Bilingual Program Participants 



ERJC 



THE SCHOOL DISTRICT OF PHILADELPHIA 



BOAXID O? EDUCATION 
JNSTIcUCTIONAI SERVICES 
PA3UC^yAY AT TWENTY-PIRST STREET 

PHILA0SLPKSA« PEMNmVANIA 19103 



MAITHEW W. COSTANZO 



ELEANOR SANDSTBOM 
tM>icroK or poiBicN lAhtcvAoa 



SUPl^aiNTtNDKNT OP tCHOOUi 



I. B2atA STAPLES 



44^-3334, 3335 



ASSCXIATB »UMW1NT»Nt>«m 
INS-niC/CTtOWAL WRVtCSS 



rfarch 3, 1972 



Dear parents: 

We would appreciate your filling out the enclosed 
questionnaire concerning the Bilingual Program in which 
your child is enrolled. 

You can mail the questionnaire in the enclosed envelope 
to Wtr. Robert Offenberg, Office of Research and Evaluation. 
You do riot have to use a stamp nor do you have to sign your 
name or your child^s name. 

This questionnaire should be mailed not later than 
April 3.. 

Thank you for your cooperation. 



Sincerely, 




ELEANOR L. SANDSTROM 
Director of 

Foreign Languages Office 



ELS:rg 



End. 
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THE SCHOOL DISTRICT OP PHILADELPHIA 

BCAKD OF EDUCATION 
n^TRUCTIOxVAL SERVICES 
PAJUCWAV AT -nCTKNTy-mST STRJSET 

PHfLAttiW^HU, PkNTOYLTANIA IJIOJ 

MATTHBW W, COSTANZO 

suMRtirniNDiKT or acHOOu 

r, BZRASTAn^S 

ASAOClATfi JUf BUNTBNMNT 
INSTRUCTIONAL MlVlCa 



BLBANOK L SANDSTXOM 

MfUKTlOK or POtllCM lAKCUAfiBS 

5535 



1 de marzo de 1972 



Estimados padres: 

5e les agradecer^ tengan la bondad de llenar el cuestionario 
que acompafia a Ista, en relaci6n ccn el Frograma Bilingue en que 
su hi jo estS matriculado en la escuela. Pxxede enviarlo por coireo 
usando el sobre que se le incltye dirigido al Sr. Robert Offenber^ 
de la Oficina de Investigaci6n y Evaluaci6n. No necesita ponerle 
sello. Tampoco tiene que firmarlo ni escribir el nombre de su hijo. 

El cuestionario debe enviarse antes del dfa 3 de abril. 

Muchas gracias por su cooperaci6n y por su atenci6n. 




f Directora 
Programa Bilingue 

ELS:rg 
End. 
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THE SCHOOL "OISTRICT OF PHILADELPHIA 



Office of Foreign Languages 
Office of Research and Evaluation 
LET'S BE AMIGOS 



PARENT'S QUESTIONNAIRE - BILINGUAL PROGRAMS 



:hool 
lass 



At home, I_ speak to nv^ ch.i.ld ; 



always in Spanish. 



(Check One) 



sometimes in Spanish^ sometimes in English 



always in English, 



At home, iTv^ child speaks to^ me : 



always in Spanish. 



sometimes in Spanish, sometimes in English 



always in English. 



At home, mj^ child speaks to others in the family: 

always in Spanish. 



sometimes in Spanish, sometimes in English 



always in English, 



Is your child studying English in school? 

NO 



YES. If ^ESj are you satisfied with the 
progress he is making in English? 



NO 



YES 



Is your child studying Spanish in school? 

NO 



YES. If lYESL are you satisfied with the 
progress he " is making in Spanish, 

NO 



iftt bUHUOt DJSTSICT OF PHILADELPHIA 



Office of Foreign Languages 
Office of Research and Evaluation 



CUESTtONARlO PARA LOS PADRES SOBRt EL PROGRAhA BILINGUE 



Escuela 
Clase 



1 . En el hogar, ^ hablo a mi h i jo(a) ; 
(Marque una) 

2. En el hogar, mi hijo me habla: 



siempre en espafiol . 

a]gunas veces en espaftol , algunas 
veces en fngles* 

s iempre en ingl es . 



siempre en espaftol . 

algunas veces en espafiol, algunas 
veces en ingles • 

s iempre en i ngles . 



3. En el hogar, mj^ hijo les habla aotros miembros de la familia: 

s Iempre en espaflol . 



algunas veces en espaftol , algunas 
veces en fngl es . 

s iempre en ingl es • 



iAprende su hfjo ingles en la escuela? 



No. 
ST. 



Sf usted ha marcado {stT, iest5 usted 
sat isf echo con el progreso que hace 
su h ijo en ingles? 



No. 

sr. 



5. iAprende su hijo espanoi en la escuela? 
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No. 



ST. Si usted ha marcado |sTj, iesti usted 
satisfecho con ei progreso que hace 
su hijo en espafiol? 

^ No . 

ST. 



6, Does ybur cnild like learning Spanish and English ir. school? 

NO 

YES 

7. Do )jou like your child to be learning Spanish and English? 

NO 

YES 

8. Do you want your child to study two lang^^uages next year? (Spanish and English) 

_N0 

YES 

9, Have you visited your child's Spanish-speaking teacher this year? 

NO 

YES 

10. Have you visited your child's English-speaking teacher this year? 

NO 

YES 

11. Have you ever attended a meeting about your child's school program? 

NO 

YES 

12. Please write any suggestions you have for the bilingual program. 
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6. Lie gusta a su hJjo aprender espano'; e ingies e.-i ia escuela? 

No, 

S I . 

7. iOesea usted que su hfjo aprenda esi>aftol e Ingles? 

No. 

sr. 

8. iDesea usted que su hijo estudie dos lenguas el afio que viene? 

_^ No. 

ST. 

9. iHa visitado usted al maestro hispano de su hijo este afio? 

____ No. 
ST. 

10. iHa visitado usted los otros maestros de su hijo este afio? 

No . 

sr. 

11. iHa aststldo usted a alguna reunion del programa de 3a escuela de su hijo? 

•_ No. 

ST. 

12. Tenga la bondad de escribir cualquier sugerencia o recomendac i6n que usted 
tenga sobre el programa b|]ingue: 
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Appendix 2.3 Outcome of 
1971 Bilingual Training Institute 



Appendix 2.3 Outcomes of 1971 Bilingual Training Institute 

During the sumnter of 1971 a training institute v;as held to prepare 
Spanish-speaking residents of the Philadelphia community to teach in the 
public schools. Thirty participanrs enrolled in the institute, two of whom 
failed to complete the program. Of the 28 who finished, 18 went on to teach 
in the Bilingual Program (one of whom resigned at midyear) and 10 taught in 
other city schools which required Spanish-speaking teachers. 

The summer institute was funded by the Title V of this Educational 
Professional Development Act. Personnel who completed the program earned 
12 college credits and were awarded emergency certification by the State of 
Pennsylvania, and accepted a commitment to continue their studies on a part- 
time basis. 
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STUDY 3- CURRICULUM DEVELOPMENT, DISTRIBUTION, AND TRYOUT IN THE 

ARRIBA PROGRAM 



One of the most critical needs of the. project has been curriculum 
materials for older Spanish-speakinq pupils, pupils in the upper elemen- 
tary, junior hiqh, and senior high school levels of the ARRIBA component. 

Objective 1.5 stated that proqrairis for these levels would be 
reviewed and where necessary nev? materials would be developed. The 
continuation proposal for the third ooerational year stated that developed 
materials v/ould be tried out and a study of pupil performance on them 
would be conducted. 

Durina the school year it was necessary to modify these goals 
because distribution of materials was inconsistent and often, rtone too late 
for the planned systematic tryout. The need for this "study became most 
evident at midyear, when it was found that only a small portion of the 
curriculum materials which had been prepared were actually in use. 

The evaluation staff and the project director then agreed to a 
a three-pronged evaluation in the curriculum area in order to gather data 
which would provide a clear assessment of the then current state of affairs 
and mitigate implementation problems: 

1. A study of the teacher's view of the curriculum-distribution 
process was made in order to facilitate a systematic assessment of pupil 
performance next year. 

2. Examination of newly developed curricular materials was 
undertaken to assess their quality. 

3. A Study ci teachers* reactions to completed curriculum mater-' 
ials was undertaken to assess (a^) the elements which went into teacher 
decisions whether to try a curriculum set or not, and (b) the teachers' 
views of ways the materials which hrd been tried could be improved. 

Procedures 

Program 

Teams of two to four teachers who understand, read, write, and 
speak both Spanish and English fluently work under the direction of the 
curriculum coordinator. They adapt (into Spanish) curriculum materials 
in use throughout the school system or prepare new instructional materials 
for the program when needed. During the summer months teachers work on 
a full-tirre basis. During the school year, the curriculum coordinator 
supervises both the curriculum-development activities of teachers who 
prepare curriculum materials on a part-time basis and teach classes and 
the activities of the full-time curriculum development staff. 

In addition to teacher-writers-, the services of tv;o media special- 
ists and an illustrator have been used in the development process. 

As materials are developed, they are to be made available to 
the teachers through the supervisor of the ARRIBA component, who is respon- 
sible for locating need for materials and planning a way of seeing tliat 
they are delivered to the schools. 
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Evaluation 



Each of the three problems explored in this paper required a 
different evaluation approach. 

Assessment of Teachers ' Views of the Curriculum Development and Distri- 
bution Process . 

This" part of the study was conducted through the use of struc- 
tured interviews. Teachers were visited at their schools by a bilingual 
member of the research staff. The teachers were asked to cite their im- 
pressions and problems in obtaining and using curricular materials deve- 
loped for the program. When each teacher's commenting was completed, 
she was asked thii specific questions shown on the interview format in 
Appendix 3.1, if she had not already provided an answer to them. After 
the interview, a summary of each teac?ier*s comments was recorded on the 
interview sheet. It was orginally planned that a cross section of all 
ARRIBA teachers would be interviewed, but time pressures and highly con- 
sistent initial findings resulted in cutting this down to 12 teachers 
at six schools . 

Curriculum- Deve lopment Coord ina tor ^ s Review of New Materials 

During the second year of operation, a curriculum-development 
checklist was prepared for assessing the degree to which developed materials 
met the curriculum coordinator's criteria for successful units. The 
current study applied the same methods to materials completed in the 
1971-1972 fiscal year: the coordinator of curriculum development prepared 
a brief description of each set of materials prepared for the ARRIBA program, 
then rated them on each of the items on the curriculum development check- 
list. 

Abstracts of the descriptions of the materials and the results 
of the checklist ratings were included in this part of the study. 

Teacher's Evaluation of Materials Distributed . 

The last part of the study was a second set of structured inter- 
views. They were conducted in March and April of 1972, and were designed 
to assess (a) what went into a teacher's decision whether to use a set of 
materials, and (b_) once a set of materials was tried, how successful the 
materials were in the classroom. The procedure was like that used to 
assess teacher views of curriculum development.' A bilingual interviewer 
visited each teacher. He invited the teacher to discuss the materials 
freely, and then if the teacher had not commented on them, he asked about 
specific points of the structured i;*iterview format. Two different inter- 
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view structures were used. The one shown in Appendix 3.2 v/as used for 
all but one set of materials. The interview formaT: shown in Appendix 
3.3 was used for evaluating junior high school "English as a Second 
Language" materials. A digest of the teachers* remarks was prepared from 
both interviews. 

Results and Discussion 

Teacher Views of Curriculum Development and Distribution Processes 

It was orgirially planned that a sample of two or three teachers 
would be interviewed at. <^ach site in the ARRIBA proaraiu in order to 
deterraine the amount of knowledge teachers had about the curriculum's 
development and to uncover ways in which the distribution process could 
be improved for the coming year. The information gathered in the first 
interviews of 12 teachers who taught at six schools in the ARRIBA conpo- 
nent suggested that further interviewing would not produce new insights. 
As a result, the attempt to interview each teacher was discontinued - and 
only four more teachers were seen to obtain data about specific curricu- 
lum materials that had been developed* These teachers were asked about 
curriculum development and distribution in an abbreviated interview. 

The first questions about curriculum development and distribution 
(Items 3 and 4 on the interview format shown in Appendix 3,1) were designed 
to ascertain the actual ways in which teachers were obtaining curriculum 
materials. Teachers were asked how they obtained the curriculum materials 
that they were using. Responses to these questions showed that teachers 
in the ARRIBA program did not have a clear idea of wh.=^re curriculiim mat- 
erials originate and how they can get materials most efficiently. 

Nine of the 16 teachers interviewed were not aware of the exis- 
tence and function of either the Curriculum Materials Center (at 219 Nortli 
Broad Street) or the Media Ceni:er at the Potter-Thomas School. Three 
teachers clearly knew the function of these centers and the curriculum- 
development coordinator, and four knew the curriculum-development coordi- 
nator; however, six teachers thought that the major responsibility of 
the coordinator was the supervision of teachers* 

Five, of the 16 teachers reported that they requested materials 
from their supervisor but never received them. Five said they did not 
make any request, becaut^e they did not knov/ what was available. Four 
teachers made requests and got the materials that they needed. The other 
two teachers apparently received all matierials without initiating any 
requests. Six of the teachers said that they were to call the project 
director if they needed any materials: Only a third of the teachers 
making requests for materials asked their supervisor, the person who was 
the official channel for requests to obtain materials. 

The interviews with teachers also suggested that materials 
distribution occurred too late in the school year. Regardless of whether 
.materials were obtained through request initiated by the teachers or 
provided by the project staff without prior request, nine of the 16 
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teachers felt that materials were not delivered in time for their use to 
best advantage (most being delivered in Novernber) . Tvelve teachers re- 
ported that too few copies of at least some materials were received for 
their needs. 

Item 5 asked teachers how curriculum-materials distribution 
could be improved next year. Item 6 asked how the teaching of the 
materials could he organized so that a fair assessment of pupil perfor- 
mance in the appropriate areas could be made, (Responses to these two 
questions have been integrated here.) All interviewed teachers stated 
that they would like to have a list of all curriculum materials avail- 
able , and directions for obtaining them, before the school year began. 
When questioned about how the books should be distributed, ail respon- 
dents stated that they felt that the project staff should deliver mat- 
erials to the schools because other non-project teachers are not required 
to go to curriculum centers or depots to obtain the materials. Some 
also said (incorrectly) that the * curriculum centers were open only 
during the hours when school is in session. 

Three-fourths of the respondents felt that before the school 
year began, a meeting should be held at which teachers could have contact 
with the coordinator of the curricultim center and subject specialists 
from outside the program, 

VJhen teachers were asked specifically about how the teaching 
pould be organized to permit a test of pupil mastery of the content of 
pro ject- developed curriculum units, the interviewer noted that teachers 
became more serious. Fourteen of the 16 teachers felt that a complete 
course outline and a primary text should be provided for each si±^ject 
during the school year. These outlines should provide an indication of 
the testing schedule and the content of each examination . The remaining 
two teachers stated that they needed to know cit least the content and 
date of the examinations. Teachers mentioned department' heads , the 
central project administrative staff, ^ind guidelines used in Puerto 
Rican schools as possible resources for developing these outlines. 

When asked for any additional comments, all 12 teachers who 
were involved in the coitplete interview commented that there was, in 
their opinion, a need for improving contact and feedback with the central 
project administration and also a need for someone whose job was to 
physically distribute books and materials. 

These data suggest that instruction in the ARRIBA component is 
loosely organized because the teachers, who do the instructing, were not 
clear about (a) the goals of the courses of study that they were teaching 
in each subject, (b) materials that they were to use / (£) the way to get 
materials, and (d) what to expect in terms of pijpil performance. 

It is not clear, at this point, how this situation developed. 
However, discussion with supervisors and the cxirriculum coordinator suggest 
that their roles and responsibilities are not clearly defined. It also 
appears that no one has a clear-cut responsibility and there is no 
definite procedure for the physical delivery of materials once they are 



41 



requested. The data suqqest thiat during the suironer of 1972, courses of 
study should be delineated and a better materials-distribution system 
should be developed. 

Curriculum Coordinator ' s Review of Newly Developed Materials 

Information provided by the coordinator of curriculum development 
showed that there were seven new sets of curricular materials completed 
by the project staff for the ARRIBA component. Two of these, Reference 
List of Materials for Teaching English as a Second Language (for use at 
all grade levels) and Brief Chronology of Puerto Rican History (for use 
in the junior high schools) were designed for use of teachers, not pupils. 
They were excluded from formal evaluation because there were no stated 
criteria against which to evaluate them. The coordinator of curriculum 
development reviewed the remaining five against criteria developed during 
the second year of program operation. Of these five, three were new sets 
of Spanish-language science materials for the junior high schools: Astro- 
nomia (two booklets) , Biologia (one booklet) , and La s Maquinas' Simples 
(six booklets) . English for Puerto Rican Students , a major revision of 
the "English as a Second Language" materials developed last year was 
also completed for use in high school classes in English as a Second 
Language. The last set of raaterials was a revision of the Muckley 
"Spanish as a First Language" units developed last year. They are now 
multimedia packets, partly zape-recorded, with accorrpanying teachers' guides. 
The titles of the five units are Recien Llegado, P^aseando En Puerto 
Rico , Gigantes Borincanos ^ Una Carta , and Un Compromise Formal . 

Table 3.1 shows the summary of the ratings made for the coordina- 
tor of curriculum development on the criteria established last year for 
curriculum- unit evaluation. As shown in the table, all five sets of 
materials were rated appropriate for the intended grade levels and student 
backarounds, had sufficient variety of activities, and made provision 
for pupils with different learning rates. Inhere it was relevant, the 
materials were found appropriate for students' previous knowledge in tiie 
subject area and were sequentially organized. As was the case last year, 
elements of curriculum design intended to enhance pro'ject accountability 
and uniformity of teaching procedures were likely to be the weak points 
in the products: four of the five lacked clear objectives and clear 
statements of intended pupil-performance outcomes , and two sets of 
materials lacked indication of pupil -evaluation procedures. Three of 
the same materials lacked a teacher guide; four did not indicate how 
necessary equipment could be obtained; and two did not include specifica- 
tions of or ways to obtain, necessary aids or materials. 

Taken together, these results suggest that/ in the curriculum 
coordinator's view, the conceptuali '.ation of the curriculum materials for 
the ARRIBA coirponent has been good, but the mechanisms for testing them 
and using them efficiently in the classroom are still not highly developed 
as they should be. 



Table 3.1 Sumniary of supervisor's ratings, on project-developed criteria, 
of materials completed this year for five curricular units.* 







Number of Units 'Rated . 


Criterion 


Yes 


No Not Applicable 



1. Appropriate for intended 5 0 0 
grade levels* 

2. Appropriate for students' 5 0 0 
cultural background, 

interest level, and 
experiential field. 

3. Appropriate for students' 2 0 3 
previous knowledge in the 

subject matter or field. 

4. Specific objectives 14 0 
clearly stated. 

5. Sequential organization 4 0 1 
and structure . 

6. Observable performance 14 0 
outcomes stated. 

7. Reasonable variety of 5 0 0 
learning activities. 

8. Evaluation procedures 3 2 0 
included. 

9. Provision for individual 5 . 0 0 
rate of learning included. 

10. Teacher guide including 2 3 0 
suggested classroom pro- 
cedures . 

11. Availability of equipment. 14 0 

12.. Aids, materials needed 12 2 

to teach unit specified, 
and where obtainable. 



It should be noted that these materials were distributed systema- 
tically. Evaluation of them by teachers was undertaken. These teacher 
reactions appear in the next section of this paper. 
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Teacher Evaluation of Secondary-Level Curriculum Uniti^ Distributed in 



1971-1972. 



Regardless of how "good" a set of curriculum materials is from 
the point of view of pupil performance, it is necessary that curricular 
materials meet the test of teacher acceptance , before they can be regarded 
as successful. To this end, interviews were corducted with teachers who 
tested packages of materials which became available for general distri- 
bution this year. Durinor the 1971-1972 school year, six sets of Spanish- 
language materials in thf. r-.reas of Spanish as a First Language, social 
studies, and science became available for trial. In addition, a year 
long oral-aural language course for beginning ESL pupils was completed 
and tried. After the teachers had an opportunity to begin to use these 
materials (March, 1972) the interviews were conducted, and the following, 
condensations of the teachers' reactions were made. 

Spanish as a First Language (Muckley) multi-media packets 
(Recien Llegado, Paseando en Puerto Rico, Gigante s Borincanas, Un Conproiru-so 
Formal, Una Carta) for Junior and Senior High School Pupils . Six teachers 
received packets of these materials from the Curriculum Center, Five of 
the teachers reported that they were using them at the time of the inter- 
view (March, 1972) in their "Spanish as a First Language" classes. One 
teacher stated that she was planning to begin to to use the materials in 
the spring, but had not yet done so because the tape-recorded dialogues 
which would accompany the written text were not received until January. 

Of the five teachers who had received the materials packages, 
three had used the entire set: one used it exactly as published, cind two 
reported making some adaptations . 1'he remaining teachers said that they 
used only parts of the curriculinn package: one did not use some parts 
because she felt there was an insufficient number of exercises, and one 
said that she had fo^md more interesting activities which could be sub^ 
stituted for parts of the materials. 

All responding teachers agreed that a major strength of these 
materials was their relevance to the experience of the target groi:p — 
migrants from Puerto Rico. Cited were appropriateness of the material 
for the pupils who could identify with characters in the stories, high 
pupil motivation because of the wide variety of activities (taped dialogues 
songs, etc.), and value for developing discussions because tiiey coirpared 
the lives of Puerto Ricans on the island and on the "mainland. 

One teacher (at a junior high school) felt that the materials 
were too siirple for most of her pupils, and that they could be made more 
challenging, especially in terms of vocabulary. Three teachers stated 
that there was no clear sequence to the materials, that they lacked conti- 
nuity. One of these three teachers felt that the lack of continuity was 
especially a problem in the grammar work built into the units. One 
teacher felt that instructions to the teacher should be removed from the 
pupil copies of the materials. One teacher felt that the taped dialogues 
should be written out, and the pupils should be given copies. One teacher 
said that the exercises were too easy and failed to reinforce the graimner 
points raised in the text. One teacher said that she did not understand 
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the purpose (teaching objectives) for which the materials had been prepared. 

Puerto R ican History , for the upper elGmenbory an d junior high 
s chool levels . Two teachers were using these materials. One teacher at 

MPPer elementary level used these materials as the primary source 
for teaching social studies. A second teacher, at the junior high school 
level, used them as a supplement* Both teachers reported using the 
entire text in the sequence that it appears in the materials. 

Both teachers felt the materials were strong presentations, with 
a great amount of intormation about Puerto Rican history. One teacher 
felt that the chronological presentation was an advantage. 

The teacher at tlie elementary level felt the vocabulary was too 
hard for the children in his class. He felt that the authors' explanations 
of some topics did not agree with those of other sources- He also felt 
thcit a package of supplementary pictures, maps, films, or slides would 
be useful. The junior high school teacher felt that the materials could 
be improved by including more discussion of important individuals in 
the history of. Puerto Rico, and by including more about contemporary 
history. He noted that use of a Spanish-language newspaper, especially 
the Sunday edition of Kl Mundo , partially filled this gap. Both teachers 
noted typographical errors. * 

Brief Chronology of Pue rt o Rican History for upper elementary 
ctnd junior high school levels (teacher^s guide). At the time of the 
survey/ one teacher at the junior high school level was using the Brief 
Ch ronology as a supplement to the project-developed text, Puerto Hi can 
History . The responding teacher stated that the material would be improved 
if more attention were given to important personalities. Errors in spelling 
were found. 

Si nple Machines (Introduction and Booklets 1-5) for the junior 
high school . Three teachers received the Simple Machines science materials. 
Two teachers reported that they used them; the third stated that he did 
not teach science in the bilingual program, and that the shipment was cin 
error. 

Of the two teachers who used the materials , one reported that 
he used the entire package of six booklets with his ninth-grade class, 
the introduction and the . first two booklets with his eighjbh -grade group, 
and the introduction alone with his seventh-grade class. The second 
teacher reported that the entire package of booklets was used with all 
his students in seventh, eighth and ninth grades. The .first of these 
teachers stated that the booklets served as the only resource; the .second 
teacher said that the pupils used the booklets but his own preparation 
was based on other resources. 

Both teachers who tried Llie materials reacted favorably, one 
stating that there were, in his op:».nion, some weaknesses, but the materials 
were the best that were available to him- Both teachers felt that the 
materials were most suitable for ninth-grade pupils, and both agreed that 
the plan of publishing the materials in separate booklets was an advantage 
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because that would permit pupils to work at rheir own pace and to evaluate 
themselves as they go. Both teachers felt tl^.at tr;e texts were generally 
free of eirors and ambiguous or misleading statements. 

The teachers felt that the materials could be strengthened by 
the addition of more deironstrations vDf concepts, especially if tlie pupils 
could carry out or try these demonstrations themselves. At the same time 
the teachers noted that, because they have no laboratory space and no 
materials, the demonstrations which already appear in the text are hard 
to carry out. 

These booklets were reviewed for content by the teachers and 
a member of the research staff who is a mechanical engineer. The latter 's 
review contradicted the opinion of the teachers that the materials were 
error-free, noting that the authors were unclear in their atterrpts to 
make a distinction between weight and mass. In his opinion further edit-- 
orial work was needed. Informal discussion between the research staff 
and the teachers r^^^gested that the major problem — that there is no 
convenient way to carry out demonstrations — could be remedied by prepara- 
tion of small, lightweight, portable sets of levers and pulleys designed 
to demonstrate the phenomena described in the course of study. 

Mi Primero and Mi Segundo Libro de Astronomia for the junior 
high school level . This material was being used by bne teacher at the 
junior high school level. Only the first book had been used (with seventh- 
grade pupils) at the time when the interview was conducted. The teacher 
did not have enough copies to use it with the eighth and ninth grades 
but felt it was suitable for those grade levels. 

The teacher felt that the presentation of the materials was 
siirple enough for pupils in the seventh grade. Other favorable reactions 
were that the pupils could work at their own pace or in groups, could 
conduct self-evaluations, and had an opportunity to read/ write, and 
draw in the booklets. At the time of the interview, the teacher did not 
h ive enough information to comment on weaknesses, other them to note, 
thai: there were some typographical errors. 

Quimica, first unit of a series for the high school . According 
to the coordinator of curriculum development, these materials were 
intended to serve as the first part of a year-long program for the high 
schools. They have not yet been used in the high schools, but one teacher 
(the author of the series) tried this introductory' chapter as an indepen- 
dent unit on the junior high school level. This teacher stated that 
he used these materials as a review resource, the material having already 
been presented from other sources. Because. the teacher who conducted 
the field trial was the author, he refused to comment on hie experience 
with the materials, other than to say he felt that the year- long program 
should be completed. Discussion with the coordinator of curriculum- devel^ 
cement revealed that he" and some of the high school science teachers felt 
that the materials were too conplex for the target group and needed to be 
simplified before wider distribution could be recommended. 
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English for Puerto Rican Stu d ents for t.he junior high school 
level. These Kiaterials were developed in the Bilincrual Program for pupils 
at the junior high school level who were having their first formal instruc- ■ 
tion in English. However, they were distributed to both teachers in the 
program (three junior high school teachers and one high school teacher) 
and a few teachers (three elementary, two junior high, and one senior high) 
who had classes of English as a Secu-id Language.. 

Of the ten teachers, five (two high school, two junior high, 
and one elementary) teachers reported that they had no" used the material 
in any systematic way. The other five reported using the materials con- 
sistently with at least some pupils in their class, although all teachers 
supplemented English for Puerto Rican students with other materials. 

Neither high school teacher reported more than a brief trial 
of the materials: one Su:.d that there was only one pupil at a beginning 
English level; the other felt that the materials were too elementary for 
her class . 

One of the junior high school teachers conducted only a brief 
trial of the materials. He said that his pupils became bored with the 
large amount of repetition, but noted that his pupils did like the 
dialogues. A second junior high school teacher said that the materials 
were not suitable for her class, because the great amount of oral work 
prevented the class from being divided into simultaneously working, 
homogeneous groups. This teacher said that the materials would be useful 
for pupils with no English if they were in a separate class. The three 
remaining teachers at the junior high level reported that they could 
use the materials in combination with other texts in heterogeneous classes. 

At the elementary level, the classes which used English for 
Puerto Rican Students were not part of the Bilingual Program, but were 
homogeneous classes designed for new immigrants. Their teachers reported 
that the materials were satisfactory (even for first'-grade pupils, although 
they were not planned for use with this age level). 

English for Puerto Rican Students has a unique set of symbols 
designed to give the teacher cues as to how the dialogues should be pre- 
sented. A few of the responding teachers stated that the symbol system . 
was annoying^ but none said that the problem was serious enough to prevent 
use of the materials. When asked about the organization of the material , 
three teachers commented on its rigidity, i.e., that it prescribed too 
much of the teacher's behavior. One of the three noted, however, that 
this was a problem becau e of the heterogeneity of the classes with which 
she worked. Other teachers had no opinion or^^felt that the highly struc- 
tured approach in these materials was an advantage * 

All teachers felt that the presentations of pronunciation, 
grammar structures , vocabulary , and comprehension were good for beginning 
pupils. The word games were singled out by one teacher (at a junior high 
school) for special praise. One teacher noted that the pronunciation 
drills provided only minimum pairs. She felt that sentences and more 
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conolete utterances v:ould be desirable additions. 

One teacher noted that the pupils liked the use of Spanish 
names and the content about Puerto Rico. Most teachers at the secondary 
level indicated that the materials they used along with English for Puerto 
K ican Students compensated for its lack of readinq and writing activities. 

Overall, the interviews with teachers suggested that the useful- 
ness of these materials depended heavily on two factors: the pupils exposed 
to the materials must be having their first contact with English instruc- 
tion, and the class must be organized so the teacher can give a sxabstan- 
tial block of her time to the pupils using the materials. 

Conclusions 

Taken together, the findings of these studies indicate that, 
while some criticism of each of the" developed curriculum materials is 
possible, the major problem has been in the area of materials distribu- 
tion. The. suggestions gleaned from some of the teacher interviews 
regarding the prescription of materials to be studied by the pupils, 
preparation of course outlines, and inprrved supervision reflect major 
needs in the project. They must be fulfilled if the investment in curricu- 
lum development is to pay dividends. 
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Appendix 3.1 Structured Interview for Assessing 
Distribution of Project Developed Curriculum Units 



LET'S BE AMIGOS 



Title VII Bilingual Program 

Research and Evaluation 

Foreign Languages 

Structured Interview of Teachers Using 
Program Developed Units, 
part I Curriculum Distribution 



1. Identification: 
School 



Teacher's Name 



Grade Level taught 

Interviewer 

Date 



3« Find out which project developed materials the teacher is using in each 

subject that he teaches. r'Nj 

Subject Title Author 



3. Find out from whom the teacher got the materials. / 

Note: (If the teacher does not mention supervisor, the school itself and the 
curriculum center, ask specifically about them). 



Note ; (If it is not yet clear, find out whether the teacher knows about the 
Curriculum Development Center at 219 N. Broad, Richard Krough, and 
the Materials Center at Potter-Thomas). . 
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4* 



Did the teacher request any materials? If so, what did they ask for, 
whom did they ask, and did they get them? ' 



5. How can we improve the distribution of materials in general for next year? 



6 m tfext year we would like to examine pupil performance on some of the materials 
which have been written for use in the project, Kow can we distribute 
those materials,, and what kind of support can we give^ to assure that they 
get a fair trial? ^ 



7. Anything else about curriculum materials distribution, that we should know? 



March, 1972 
/dsb 
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Appendix 3.2 Structured Interview Format For Assessi 
Curriculum Units Tried by Teachers: General Form 
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LET'S BE AMIGOS 



Title VII Bilingual Program 



Structured Interview of Teachers using 
Program Developed Units 
part II Asse^ssment of Units Examined and Tried 



Units or Set of Units Being Reviewed; 



1. Find out whether the teacher (a) only examined the materials^ (b) tried 
the materials^ but did not use them completely^ (C) used the materials 
completely, ^ 



2. Find out what went into that decision to try or not try the materials. 

Probe for (a) elements of program management or supervision^ (b) characteristics 
of the materials themselves, (c) elements of the classroom situation* 



3. Find out with whom the materials were used. 



4. Based upon the teacher's experience, find out what is good about the 
materials. 



5* Based on the teacher's experience, find out what needs to be improved. 



6. Find out whether these materials were used as a main text or as a 
O supplement and why. 



Grade level 



Age 



Children's educational background 




> 1 m : If these materials are going to be revised next year, could the teacher 
think about any changes to improve them? Especially, ask for instances 
of mis information, ambiguity, etc. 



8* If the teacher, after looking at the materials decided not to use tbem^ 
find out reasons. 



9. If teacher is not specific in answering question find out if it was 
either because of late delivery or because of properties of materials. 
If properties of materials, what were they? 



10. What changes can the teacher suggest (either in the properties or dis- 
tribution of these materials) that could make them more likely to be 
used next year? _^ \ ' 



11.. Is there anything else about these materials that the staff for curri- 
culum development and distribution should know? 
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March, 1972 
/dsb 
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Appendix 3.3 Special Structured Interview for English 
for Puerto Rican Students Curriculum 
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^LST'S BE A>iIGOS 



OJitle VTI Bilingual Program 
Research and Evaluation 
Foreign Languages 

i 

I Structured Interview of Teachers; 

/ English for 'Puerto Rican Students 

* 

I 

Identification - Teacher • 

Grade levels taught . 

Interviewer 

Grade levels in which raaterials are actually in use: 



a. If the materials are in use in only some grade levels/ find out why 
they are being used in some, but not in others: 



b. If the materials are not in use in any grade levels, find out why 
the teachers have not used them. 



c. If the teacher has not already expressed an opinion, find out how the 
materials relate to each of the following: 

(1) Amount of English pupils Icnew before the program began 



(2) Age level of pupils 



(3) Grade levels of (e.g., sixth grade, seventh grade) the pupils 



A- 



(4) Presentation of materials in the book, especially the symbol system 
materials or instructional aids needed 



(5) Organization of the activities into a highly structured course of 
study ^ 



(6) T!he adequacy of the content and/or presentation of: 

(a) Pronunciation 

(b) Grammatical Structures 



(c) Vocabulary 



(d) comprehension of English utterances 



3# If the teacher has used the materials find out if they were used as a 
supplement or as the main text. . 



. a. List any other materials the teacher used. Indicate which were used 
most of the time^ and which were used occasionally. 



b. Ask the teacher why these other materials were useful (or better) than 
English for Puerto Rican Students. 



4. Ask the teacher how the English for Puerto Rican pupils materials can be 
improved, if they are re-edited or revised. 



ER|c*^ry, 1972 
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STUDY 4. FOLLOW-UP OF ELEMENTARY -LEV3L '^ARRIBA" PbTILS WHO MOVED INTO 

REGULAR EigGLISH-LANGUAGE CLASSES 



Durinq the third year of program operation, a discrepancy was 
uncovered botiwcen project policy and the intentions of some school personnel 
in the ARRIRA program. VJhile central project administration felt that 
most pupils at the elementary grade levels needed to remain in the ARRIBA 
program for several years before they were ready to do well in regular 
■English-speaking classrooms, some local school personnel felt that many 
pupils were ready to enter English-speaking classes after only a year in 
the ARRIBA program. 

This study was undertaken to begin to assess the relative validity 
of the opinions expressed by the two groups within the project. As it 
was not included in the original evaluation plan, the study must be con- 
sidered an informal assessment. For convenience, it was confined ro two 
grade levels — pupils who were in the fifth and sixth grades in 1970-1971 
and those who were now enrolled in the sixth and seventh grades. 

Procedures 

Subjects 

At the end of the 1970-1971 school year, 21 pupils were enrolled 
in the. fifth grade and 21 pupils were enrolled in the sixth grade of the 
ARRIBA program. These pupils were followed up for this study. 

Procedure and Instrument 

A structured interview format was developed. It is shown in 
Appendix- 4.1. Current teachers of the pupils who had moved to regular 
Philadelphia classes were interviewed by a member of tne research staff. 
Most interviews were by phone, although if it was convenient a face-to- 
face meeting was arranged. The procedure was to ask the teacher to 
discuss generally the perfomance of each pupil. The interviewer then 
continued with the specific items of the interview schedule if the teacher 
had not already provided clear information about the nupil in each area. 
The interviews were conducted in February and March, 1972. 

Analysis 

Results were tabulated and, where appropriate , percentages were 
computed. Frequently, the tabulation is based on a synthesis of the 
interviewees* responses to several questions. These synth*Jdes were made 
by the interviewer ♦ 

Resul ts 

Results of the interviews with teachers who had Spanish-speaking 
pupils in their classes are summarized on Table 4.1. of the 42 pupils who 
were in the project pupil file during 1970-1971, 13 could not be located 
during 1971-1972. The ■ remaining 29 pupils were either in the Arriba 
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program, in regular classes, or had been transferred out of the Philadelphia 
school systc^m. 

Table 4.1 Current Instruction of Pupils in AKJ<IBA 5tli and eth 
GradGs During Fiscal 1970--1971. (All pupils had 
received one or two years of instruction in the 
Bilingual Program.) 



Grade Level, 
1970-1971 

■ 


Pupils Still 
in ARRIBA 

N % 


Pupils in 
Reg. Philai 
Classes 

N % 


Pupils Transferred 

to other School 
Systeir^ (Inc. P.R.] 
N ^6 ■ * 


No. i 

Info. 

N % 


Total 

N % 


5 
6 


5 24 j 11 52 

6 29 . i 2 IC 


— — — ' 

0 0 

5 Z4 


5 24_ 
8 38_ 


21 100 
21 100 


Total Pupils 
Located 


: 11 li 


i 13 45 

{ 
I 

I - 


5 17 




29 100 



Of the 29 pupils for whom data was obtainable, 38% continued 
in ARRIBA, 45% were in regular classes, and 17% (all of them sixTJ:i-grade 
pupils) either moved to Puerto Rico or entered parochial and other private 
schools. Follow-up was possible on 12 of the 16 pupils in regular classes 
of the Philadelphia school system (11 children) and the local parochial 
school system (one child) . 

Table 4.2' shows the project-evaluation staff's judgment of 
descriptions the teachers gave of pupils who came from the bilingual 
program to their classes. The classification was made on the basis of 
teachers answers to three questions: (a) "How well is this- child doing > 
in comparison to 6ther children in the class?" (b_) "Is the child's 

Table 4.2 Teachers' General Impressions of the Pupils' Overall Perfor- 
mance: Pupils in Regjlar Classes of the Schools 





Doing Very 
Well 


Doing Satisfac- 
tory Work 


Evi dencing 
Serious Problems 


Total 


N 


1 


6 


. 5 


12 


% 


8 


50 


42^ 


100 
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.Enolish competence,- adequate for all subject areas?" and {c) "Are there 
any r-;i±) ;iects.-whe.re /'the child appears to have specific problems?" If 
the pupil's tc-acner-r reported outstanding work, or no weakiiosses , he 
wrijn f;.l rjjjsi fiod as doing very well . if the pupil was described as typical, 
or doinq satisfactory work with some weaknesses, he was put into the 
generally satisfactory category. If the teacher reported many problems 
or indicated that problems were, in her opinion, reflecting serious 
deficiences, the pupil was classified as evidencing serions problems. 
The results suggest that about half the pupils were seen as doing very 
well or doing satisfactory work, but 42% were seen as having clear 
problems. Table 4.3 has a list of the problems mentioned for the satis- 
factory and serious problems groups. 



Table 4.3 Problems listed by teachers for pupils in their classes (N = 12 
pupils), when compared to other pupils in their class. 



Problem 



Number of Pupils Mentioned 



Excellent or 
Doing Satisfactory 
Work 
(N = 7) 



General Linguistic 
Problems 

Hard time express-- 
,ing himself or under- 
stcinding. 

Problems in Pronun- 
ciation 

Subject-Area Prob- 
lems 

All/Most Subject 
Areas 

.Language Arts/ 
Reading 

Social Studies 

Mathematics 

Emo ti onal/ In te 1 1 i- 
gence Problems 
Low I. Q. /slow 

Shy, Introverted, 
■ Lacking Confidence 

Miscellaneous (Family) 
Makes him babysit, 
resulting in much 
r jsence 



3 

-1 
2 
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Evidencing 
Severe Prob- 
lems 
(N = 5) 



3 
1 
1 



Total 



6 
2 
3 
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These results, taken as a whole, point, to language -base prob- 
lems. While only' 25% of the children were reported havirg problem.- in 
self-expression or understanding, teachers seem to be saying that t^^e 
group is shy (41%) and having difficulties in reading and language arts 
(50%) or in all suject areas (41%) . All of these can be symptoms arising 
from problems with the English language that teachers do not recognize 
as such * 

Ten of the 12 teachers said they knew enough about the children 
in their classes to respond to the question, "Do you feel pupils would 
perform better in a bilingual class?'' Five of the ten responding teachers 
felt trtat their pupils would do better in a bilingual class and gave 
specific reasons: it would provide opportunity for the child to strengthen 
his English competence, and it might help because the pupil was behinc 
academically. The remaining five teachers felt that their pupils would 
not be better off in bilingual classes. Four stated that it would destroy 
pupils' self-confidence and therefore slow, their academic progress. Two 
of these felt that some bilingual utaff would be helpful in aiding the 
pupils in making the transition. One pupil was reported as performing 
adequately in her present class. 

The last question asked for general comments. The one noteworthy 
comment was made by four of the 12 teachers (33%r who said that problems 
they observed or reported were probably due to readjustment from one type 
of class to another. 

Discussion 

The project management has long maintained that Spanish -dominant 
pupils need years of contact with bilingual programs before pupils are 
ready to. enter English --language classrooms on an equal basis w^^h Anglos, 
They believe that brief, one- or two-year contact with English as a 
Second Language or with Bilingual Education, while better than nothing, 
can not provide adequate English skills for non^Engiish-speaking pupils 
to achieve well in classrooms geared for native English speakers. The 
data gathered in this study, while based o.i an informal method of 
collection and few cases, provide some relevant information. The results 
show that although at. least half the teachers feel that pupils have 
adequate mastery of English to survive in their classes, they observe 
.severe academic difficulties in (ja) all subject areas or (b) language arts 
and reading, subject areas which are primarily depentient upon knowledge 
of language. Virtually all are seen as having some sort of handicap 
even thoug h the base line for comparison is pupils in schools characterized 
by generally low achievements These findings suggest that thf.re may be 
discrepancy between the xevels of the English--languaqe coirpetence which 
teachers feel is necessary for a Latino pupil to be placed in an English 
language classroom and the level of competence necessary for him to 
succeed in that classroom. 

' To the, exter. : that t^e function of bilingual education is to 
maximize pupil performa.^ce in all academic areas while he is learning to 
work in English (and not to maximize English leamina at "the expense of 
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everything else) these results suggest that extended contact with the 
Bilingual Program is desirable. 

The results of the study, while clear, must be treated cautiously 
because of the informality of the evaluation design. However, it is apparent 
that enough evidence exists to warrent a more sophisticated, classically 
designed study with adequate controls. 
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Appendix 4.1 Interview Format For Interviewing Anglo Teachers 
Who Have Last Year's Program Participants in Their Classes 
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_ Title VII Bilin.j'jal Program 

Research and Evaii-ation 

Foreign Languages 

Interview for Teachers of Last Year's 
ARRIBA 5 tin Graders 

Narr»e Teacher 

School Grade & Class 

I. v\tiy is this child now in an Enqlr^sh language classrooin? VVlio made the 
decision to have the child placed ia a regular class? 



Note: If the child is repeating a grade, participating in a bilingual program 
of any sort, or if tii^re is anyching -else unusual about his present. 
assignmien^V jot down an ejiplanation of what and why. 



2. flow well is this child doing in comparison to other pupils -in the class? 
(English speaking pupils if he is in a class with English speakers). 



3. Are there any. subjects with which the child appears. to be having specific 
problems? ^ 



4. Is this child's English corr.petence adequate for all his subjects? If not, 
.describe those aspects of his work where difficulties ar^e. 
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no y(.)u lliinV. tliij; chi.hi wci»il».i ; )»m' it -lji ;)».;l:t.t.*r ill he v.ci.'t* in a biliinjual 
pro^j r:..jiii -ntjduij.' 



Any other comments or points 



rch, 1972 



STUDY 5. LOG OF PiJI^IL PEPPOKMAlvIC:-: i:; '^HE %'JDEi. A KODfll. B PROGRAMS 



Pupil -par form an CO outccrres in tJ-.o Model A and Modcil B conponents 
of the Let's Be Amiqos proqrairi have beon specified in two ways. All 
objectives wore prepared in the forrr» of conrprehensive starements of pupil 
performance, and TOst objectives were also described in terms of micro- 
scopic bits of behavior, called microob jecti ves . This paper examines 
pupil performance on the microob jecti ve level. 

In past years, data on each microob jective and summaries for 
the objectives as wholes were preser»ted. As the program continued to 
grow, the amount of information protiuced as microob jecrive data became 
oveivhelming. To keep the microob jective study within reasonable limits 
in the third year of program evaluation, it was decided that for those 
parts of the program operational for at least one previous year, reporting 
would be limited to summary data (although the testing was still of 
individual microob j ecti ves) . More detailed presentation, including micro- 
ob jecti ve--by-microob jective analysis of outcomes, would be presented for 
ttxe "cwo new program levels of 1971-1972, the third grade and the special 
first grade. This special first qrade accommodates the needs of pupils 
with prekindergarten and special all-day kindergarten program experience. 
The third grade was added as part of the planned upward cycling through 
the grades . 

In addition, for the first time, sufficient control of the 
Model B Anglo program has been attained so that these pupils could be 
tested in their second tongue witJi the log instrument. 

The following objectives specified that pupils would be able 
to carry out the indicated percentages of the microob jecti ves in the 
various skill areas: 

Prekinde rgart en 

1.1 Communications skills in the first language for prekin- 
dergarten (Spanish or English) : 90% 

1.2 Communication skills in the second lanq-uage for prekin- 
dergarten: Latino children in English, 90%; Anglo children in Spanish, 
80%. 

1.3 Number concepts in th.e prekindergarten: Some skills were 
to be learned to a 90% criterion level, others to at least a 60% level. 

1.4 Natural and biological phenomena for the prekindergarten: 
Sixty percent of the children would. show at least 80% mastery, for per- 
formance of at least 48% overall. 

Kindergarten 

2.1 Communication skills in the first language in the kinder- 
garten program (English or Spanish): 90%. 
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2,2 Corrjrscaiiccition skills in the second languacre in the kind(?r- 
qarten proarain: Latino children ir. I'^nglisih, 90%) Anglo, children in Spanish, 
80%. ' . . " 



2.3 Nuinber concepts in the kindergarten: 90%. 

2.4 Natural and biological phenorrtena in the kindergarten: 90%. 

In addition to the regular kinderaarten , an all-day kindergarten 
is operated in Model A. The objectives of this kindergarten program are 
similar to those of the other kinderaartens , but are enriched with additional, 
more difficult microob jectives . In the Results section of this report 
these enriched objectives are designated with asterisks. The criteria 
are the same as for the parallel objectives in regular kindergarten classes. 

First Grade 

3.1 Communication skills in the first grade in the mother 
tongue (English or Spanish): 90%. 

3.2 Communication skills in the first grade in the second Ian- 
■guage: Anglo pupils will show a 90% mastery level ;j Latino pupils will 

show an 80% mastery level. 

3.3 Number concepts in the first grade: 90%. 

3.4 Natural and biological phenomena in the first grade: 80%. 

3.7 Reading and writing in the- mother tongue: 90%. 

In addition to the regular first grade, a special first-grade 
class for alumni of the all-day kindergarten was operated. These children 
were tested with an enriched set of microob jectives , indicated with aste-- 
risks in the Results section of this report. The expected levels of 
performance (criteria) are the same as the criteria for' the regular first- 
grade classes. 

Second Grade 

5.1 Reading in the second grade in the mother tongue (English 
or Spanish) : 90% . ' ■ 

5.2 Poetry in the first and second languages: 75%. 

5.3 Writing in the first language: 85%. 

5.4 Spanish as a second language (Anglos only): 80%. 

I 

5.5 English as a second language (Latinos only): 90%. 

5.6 Social studies in the first language (English or Spanish): 
90%. 

5 . 7 Science in the first language : 90% . 
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5.8 



Mathematics: 90% 



Reading in the mother tongue (English or Spanish) : 50% 
1; 80% Level 2; all pupils should be able to read Level 1, 
established 'was not more than four errors in 10 consecutive 

6.2 Reading in the second lanauaqe : Anglos (in Spanish), 

Level 1 40%, Primer 100%; Latinos (in English), Level 3, Part 1 10^-, 

Level 2 50%, Level 1 100%, Criterion is not Tore than three errors in 
five consecutive sentences. 

6.6 Spanish as a second language: 75%. 

6.7 English as a second language: 75% 

6.8 Social studies in first language: No criterion specified. 

6.9 Social studies in second language: No criterion specified. 

6.11 Arithmetic in first language: No criterion specified. 

6.12 Arithmetic in second language: No criterion specified. 

Procedures 

Program 

Teachers in the h^odel School programs were to treat the micro- 
objectJLves as a skeleton for their teaching which they fleshed out by 
developing daily lessons and activities. Supervisors observed the class- 
rooms and reviewed the teachers^ plans. This served to assure that 
teachers were including materials designed to lead to the Model School 
program objectives, and that good teaching practices were employed. 

The general pattern was to teach concepts in the mother tongue. 
A select subgroup of topics was then retaught in the second language. 
Inhere appropriate, oral control of skills was developed before reading 
and writing were introduced. All teaching was by faculty who were native 
" speakers of the tariqet language. 

/J 

In Model A, teaching responsibilities were shared by thu teachers 
who formed teams. In Model seccnd-language skills were taught by 
itinerant specialists. In Model A, and in Model B Latino classes, pupils .. 
were under project control. In Model B Anglo classes, only "Spanish as 
a Second Language" instruction was urider direct control of the project 
staff. 



T hird Grade 

6.1 

Level 3, Part 
The criterion 
sentences . 
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Evaluation 



I nstrurr^ent > The log chr*ckiists used for prekinderaarten through 
second grade during 1971-1972 v/cri: ussontially the same as those used durini 
the previous year, with one modification. Hach log consisted of a list 
of grade-appropriate rriicroob jectives which students were asked to carry out 
Last year the students^ success was; recorded on "Digitek" sheets for 
machine processing* This year, responses . were recorded on the log sheets 
themselves, and then were keypuriched into cards. 

Two new logs were prepared for 1S71-1972, one for the special 
first-grade class added to the program and one for the new third-grade 
level. These logs are in Appendix 5.1, and can serve as models for the 
logs of other grade levels as well. In the third grade log one unique 
item tested reading. Rather than assessing individual unitary skills, 
this item asked pupils to read aloud from their reading text. Teachers 
were asked to;. skip ahead or backward in the text until a point was found 
where the pupil could read with no., more than four mistakes in ten consecu- 
tive sentences in the first language and three mistakes in five sentences 
in the second language. 

i . - ■ 

Subjects . A one-sixth sanple (about five pupils) was drawn at 
random from the Anglo and -trarti-BO groups of pupils in each team's classes. 
All pupils were eligible to be drawn (including theose who teachers claimed 
were having extraordinary difficulties) . However,' if a pupil had been 
drawn with less than two months' experience in the program, he was replaced 
by another student. 

Method • 

Virtually all pupils were tested by teachers who were part of 
the team with which they were working. A few pupils were tested by super- 
visors, when a teacher seemed to have difficulty in completing the task. 
Mother -tongue and second-language objectives were tested separately, 
by an adult who was a native speaker of the target language. 

Pupils were tested individually. The testing was carried out 
twice—once in February and once in the last weeks of May and the first 
week of June . 

A bilingual member of the research staff conducted a validity 
check, in which two pupils — one Anglo and one Latino — ^were drawn from ' 
each team's svibjects and retested on five or six microobjectives . When 
conducting the validity check, this member of the research staff did not 
have any information regarding the performance of the pupil in the teacher-*.* 
testing. 

Analyses ■ ' 

For those grade levels which have been part of the program in 
previous years, the mean percentage of the microobjectives correct was 
conputedr 
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Mean Percentage Correct = ^ co 

Mo . N 

where is the sum of the number of correct responses 

that the tested pupils made to objective 0, is the 
number of microobjectives in the objective, and N is the 
number of piipils tested. 

Similar analysis. was carried out for the objectives of the pro- 
gram's grade levels that were operational for the first time this year — 
special fii.\st grade and third arade . In addition, a graphic display of 
microobjoctive-by-TT.icroob jective performance was compiled for these two 
new groups . ' - 

Tfio cb joctivos , f rst- and second-language reading in the third 
grade r v;ere treated differently. The point in the reading series at which 
the pupils could read at the criterion is crraphed. Pupils were asked to 
read tha passage vmich the teacher felt was the most difficult the pupil 
had mastered. Xf the pupil had made fewer than the number of errors 
indicated in the criterion, the teacher moved ahead three pages, and read 
again. If the pupil had made too many errors, he moved back three pages. 
The process was repeated as many as four times until the pupil reached 
the criterion number of errors. If the criterion was not met in four 
trials, the pupil was assigned, a score of pages above or below the last 
part of the text that was read in the last trial (depending on whether 
the reading level was better or poorer than that which had last been 
attempted) • . . 

Results 

Prekindergarten , 

„ Results of tlie testing of Model A prekindergarten pupils are 
summarized in Table 5,1. Except for the. areas of second language for 
Anglo and Latino pupils (Objective 1,2) and number concepts for Latino* 
pupils (Objective 1,3) performance of pupils met ox. exceeded the year 
criteria at the first observation period* By year end, pupil perfor- 
mance exceeded the levels specified in the objectives in all areas. 
This suggests that review of the objectives and microobjectives of the 
prekindergarten coiT|)Onent is warranted. With the exception of the 
second-language area, the course of study for the prekindergarten compo- 
nent might be enriched by introducing more complex skills. 
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Table 5»1. Mean Percentage of Model A Pupils Succeeding on the Micro- 
objectives of the Prekindergarten Level. 



Anglo Latino 
Objective Midyear Year End Midyear Year End 
N = 10 ri = 8 N = 7 N = 12 

1.1 Corrununi cation Skills, 
First Lcinguaae 

(33 irdcroohjectivGs) 94%* 90%* 92%* 94%* 

Criterion: 00% • 

1.2 Communication Skills, 
Second Language 

(17 microobjectives) 48% 59% 75% 87% 

Criterion: 80% Anglos, 

90% Latinos ' ^ 

1.3 dumber concepts 

(16 microobjectives) 81%* 78%* 69% 88%* 

Criterion: 75%** 

1.4 Natural Phenonen 

(21 microobjectives) 62%* 73%* 85%* 92%* 

Criterion: 48% 



*Performance at or above criterion. 
**Nine microobjectives carried a criterion of 90%; seven had a criterion 
of 60%. • ' . 



Regular Kindergarten 

Results of sampling the regular kindergarter; class performance 
of Model A pupils are shown in Table S.2 , where it can be seen that pupil 
performance was close to expected levels of performance at year end. With 
the exception of Anglo pupil perfonucince in number cciicepts (Objective 2.3) 
performance was above or close to expected values . This is despite the 
fact that some of th^,.iqst promising pji^ils — talented alumni of the 
previous yearns prekindergarten — have been excluded from this group since 
the first year of program operation. There is , in fact, some evidence 
that the kindergarten program could be enriched in some areas. Comparison 
of tiie midyear and year-end results shows only tiny amounts of growth in 
communication skills in the first language (2.1) and natural .phenomena (2.4) 
from the midyear observation to year end, suggesting that mastery of 
inicroobjectives in these areas requires less them one full year's work. 
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Table 5.2 Mean Percentage of Model A Pupils Succeeding on the Kindergarten 
: Microobjectives 



' . Anglo Lati pq 
Objective Midyear Yparf End " Midyear .. Year End 
N = 1&^:^^N = S' N = 20" N = 20 • 



2.1 CoiTjnuni cation Skills 
First Languaq^^^'-"'^ . 



(37 mpspe;i5S5GCtives) 86 88 92*- 96' 

-eUltrrion: 90% , 



2.2 Communication Skills, 

• Second Language (17 - • 

microobjectives) 65 81* 77 . 88 

. Criterion: 80% Anglos , 

. 90% Latinos - . .. 

2.3 Number Concepts 

■ (14 microobjectives) 52 71 70 89 

Criterion: 90% • . / 

2.4 Natural Phenomena 

(16 microobjectives) 80 ■ 85 81 95* 

Criterion: 90% ^ * 



*Meets or exceeds the year-end criterion for the objective. . 

All-Day Kindergarten / 

Early ma;-3tery of year-end objectives also characterized the 
all-day kindergarten class in the Model A component. Pupils enrolled in 
this class met for a full instructional, day (in contrast to the half day 

i of the regular program) *. These pupils were selected from among' those 
who completed the prekindergarten program. The microobjectives of this 
level included some which normally were part of the first-grade curricu- 
lum and a greatly enriched set of skills in the second language. 'The 
results of the testing, shown in Table 5 ..3 , ^ show, that despite this enrich- 
ment, the all-day kindergarten program is too simple for pupils enrolled 

■ in it in three of the four subject areas, i In communication skills in 
the first and second languages (Objectives ' 2:1* and 2-2*) and number 
concepts (2. 3*) / pupil performance exceeded -:the' year-end crit::;£ion by 
the middle of the school year. The fourth --.objective , natural phenomena/ 
(2,4), was not attained, and there was no clear progress from midyear to 
year end. ^ (Not too much should be made of the small variations due to 
the small numbers of pupils observed) , 



ERIC 



Table 5.3 . Mean Percentage of Model A Pupils Succeeding on the Objectives 
of the Special All-Day Kindergarten Program 



Anglo . ; Latino 

Objective Midyear Year End -Midyear Year End 

^ N = 5 ' N = 4 ■ N =r 5 N = 6 



2.1 Communication Skills/ 
■ First Language 

(55 microobjectiv^s) 
Criterion: 90% 


99%** 


99%** 97%** 


96%** 


2.2* Communication Skills 
Second Language 
(27 microobjectives) , {y 
Criterion: Anglos 80% 
T^•l»•I»-^/-^<? ona- 

Liamnos z?u^ 


82%**' 


94%** V. 96%** 


93%** 


2,3* Niomber Concepts 

(14 microobjectives) 
Criterion:'- 90% 


93%** 


93%** 94%** 


90%** 


2-4* Natural Phenomena 

(19 microobjectives) 
■Criterion: 90% \, 


76% 


79%^ 86% 


V 74% 


*Enriched objective 
**Meets or exceeds the year 


criterion 


of the objective. 




Regular First Grade ^- 









The first grade was the lowest grade level ir>, which classes 
were conducted in both Model A, and Model B program-.?... This fact is reflec 
:ted in Table 5.4, which shows the performance of the Anglo and Latino 
groups in the two programs. The Anglo pupils in the Ludlow f and Bethune 
Elementary School#v7ere tested on their skill in Spanish as a Second 
Language, but not in other sub ject areas , because this was tlie- only 
subject area in- v;hich ■ teachisrs were obligated; to follow program-specxf ic 
courses of study . '''' .^"^ ^ " % 

The Model A fi'rst-grade group was select, in that those pupils 
who wereV most talented and who had experience in the all-day kindergarten 
v/ere .finrolled in special first grade. Only puioils without tliis si>ecial 
achievWnent and background we^ in tlie regular v, first ^grade in Model A. 
The Moc?el B, group v/as not select. ' ;/ . \, . ' ' 

"■^ef:all performance was below the lo.ye Is anticipated by tne 
program planners / with the exception of communication skills (Objective 
3.1) in the first language (in all groups) and Latino Model B pupil 
performances in number concepts (3'. 3> ^ and natural phenomGna (3:4)-, aild ■ 
reading and'^Wl:;-i ting (3.7) v;hich were at or close to -the specified levels 
'■' of performance. The less- than-anticipated , performance in the r'v.maining.- 
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Table 5.5- Mean Percentage of Model A Pupils Succeeding or. the Micro 
objectives of the Special First Grade. 



Anglo Latino 







Objective 


Midyear 
N =■ 5 


Year End 
N = 5 


Midyear 
N = 5 


Year End 
N = 5 


3. 


1* 


ComiTiunciation Skills, First 
Language (33 microob jectives ) 
Criterion: 90% 


92%* 


97%' 


98%* 


95%* 


3. 


2* 


Poetry, First and Second 
Languages (4 microob jectives) 
No Criterion 


60% 


60% 


75% 


65%' 


3. 


3* 


Number Concepts in First and 
Second Languages (13 micro- 
objectives) Criterion: 90% 


80% 


89% 


95%* 


88% 


3. 


4* 


Science in First and Second 
Languages (23 microob jectives) 
Criteilon: 80% 


50%" 


■ 77%: 


82%* 


89%* 


3. 


5* 


Social Studies, First and 
Second Languages (32 micro- 
objectives) Criterion: 90% 


88% 


87% 


90%* 


90%* 


3. 


6* 


Secbnd Language (22 micro- 
objectives) Criterion: 
80% Anglos , 90% Latinos 


83% 


81% 


89% 


98%* 



**Indica*.es performance beyond anticipated level. 
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objectives was coupled with a lack of :lear grbv;th frorr, rriidyear to year 
oncl in some suhject areas; .secpnd i^ir.quaqe (3.2/ for Model A Anqlos, 
Model A Latinos, Model B Latinos; natural and biological ' sciences (3.4) 
for Model A Latinos; and re.':^ding and "writing {3.7) for Model A Anglos - 

The general tr{5nd in '^^lese results has Leen for Mode ^ A first- 
grade perils' to have performed lees well than those of Model B. This 
was prob;JDly due to the frenoval of some of the best prepared and talented 
students from regular fir£*t grades and their placement" xn the boecial 
first grade. The objectives were planne-: and the criteria set prior 
tn the initia-cion :^f . iliis special first grade. 

Specxal First .Grade 

Table 5.5 shows the perforraance of uhis select g:roup of students,' 
most of whom had participated in the special all-day kindergarten the 
year before. The objectives of this program included most of those for tlie 
regular first grade, plus enriched activities with more second-language 
content. As can be seen, the performance by year end was '^t or very close 
to the levels stated in the objectives. At some other levels, compari- 
son of midyear and year-end results suggests that in some ::>uhject areas 
the microobjectives specified for these~cl"a'S'3e's"lhay be too easy, as 
the objectives were attained early :n the year. Fir Sublanguage communica- 
tion skills (Objective 3.1*), number concepts (3.2*), social studies 
(3.5*) and second- language (3,6*) performance were at or close to the 
criterion both at midyear and at year end. One objective, poetry (3.2*) 
did not have a criterion. 

Results were not always what would be expected. For example, 
pupils did not show cli3ar->:ut superiority in their first language (as 
.compared to their performance in the second language) in number concepts 
(Objective 3.3) , but the performance of Latino oupils was virtually the 
same in the two languages, and that of the Anglos superior in the second 
language to that of the first. 

In most other objectives, performance of the Anglos was better 
in the f/irst language than in the second, but that of the Latinos was 
similar in both the mother tongue aiid the second language. 

Because tiie Special First grade program was tried during 1971- 
1972 for the first time, microobjective-by-mi.croob jective presentation 
of the skills assessed appears in Figure 5 .1 , ' to permit review by curriculum 
developers. Table 5.6 summarizes the performance of pupils on those 
objectives which had both first- and second-language components. 

While these results must be viewed cautiously (they are based 
on relatively few cases) it appears that at least for Latinos in this ^ 
select group, skill in the second language approaches that in the first. 



lable 5.6. Year-IKnd Performcnce of Mqcio-l A Special-First-Grade Pupils 

in the Mother Tongae anu Scccnr3 Language on Those Objectives 
Whiv,i. contained Mixture,s of First- and Second- Language Skills-, 



Objective 


^•c■Lher Tongue 

No. of * 

Mi c rbob j e c ti ve s S ucces s 


Second 

No, of 

Microob jectives 


Language 

% 

Success 
* 






Analos 


(N=5) 






Objective 3.2* 
Poet3ry 


1 




60 


3 


60 


Objective 3,3* 
Nuniber Concepts 


9 




86 


.4 


97 


Objective 3.4* 
Science 


13 




85 


10 


68 


Objective 3.5* 
Social Studies 


28 




91 


5 


64 






Latinos 


(N=5) 






Objective 3.2* 
Poetry 


1 




80 






Objective 3.3* 
Number Concepts 


9 




91 


4 


90 


Objective 3.4* 
Science 


13 




89 




. 88 


Objective 3.5* 
Social Studies 


28 




91 


5 


84 



Second Grade 

' The second-grade program also was operated in botii >^odel A 
and Model B. In this instance, no special- subgroups were delineated 
and the results shown in Table 5:7 are based on samples of all second- 
grade pupils in the program- ■ (Data for the Anglo Model B pupils at 
midyear does not appeai: due tb administrative error; it is not clfer 
whether the evaluators neglected to have the teacher carry out the test- 
ing or the teacher failed to return the tests to the evaluation staff.) 
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Overall, the' results show uneven performance; with sone objec- 
tives being achieved early in the year and others no'c attained at either 
midyear or year end. All pupils v/ere able, to perform at criterion levels 
on the first-language reading skills (Objective 5.1) both at midyear 
and at year end. 

The objectives for poetry in the first and second languages 
(Objective 5.2) was not attained by any group in Model A, but Model B 
Latinos attained it at midyear and were close to the criterion at 
year end. Writing (Objective 5.3) was mastered at year end by only 
one group. Model A Anglos , but other groups were fairly close at year 
end. Comparison of writing performance at midyear and year end shows 
small but consistent gains from midyear to year end, with the year-end 
results close to the level specified in the objective. Model A Latino 
performance in English as a Second Language (5.5) showed a substantial 
gain from midyear to year end but performance was not up to the 
expected criterion. In Model Latino performance was close to the 
criterion at midyear but there was no gain from midyear to year end. 
In Spanish as a Second Language (Objective 5.4) t Anglos showed growth 
from midyear to year end in Model A; the year-end result was close 
to that specified in the objective. Model B pupils tested at year 
end performed slightly less well than those of Model A; however, the 
small number of pupils observed suggests that the difference was probably 
due to chance. 

Results in social studies (5.6) showed that Anglos in Model 
B both performed at the criterion levels at midyear and year end. Latino 
Model A second graders were nearly at criterion levels at midyear, but 
then fell back at year end. 

The pupils* knowledge of science concepts (Objective 5.7) was 
markedly different in the Model A and Model B components. Neither Anglo 
nor Latino Model A pupils were close to rhe criterion at cither midyear 
or year end (and Latino pupils regressed)'. Model B Latinos, on the 
other hand, proceeded to move ahead from midyear to year end and 
attained the objective.- 

Second-grade pupil performance in mathematics was mixed. 
Anglo Model A'pupils showed some growth .from midyear to year end, but. 
pupils stayed far below the expected criterion. Latino Model A pupils 
observed at midyear were nearly at the criterion, but those observed at 
year end had regressed substantially. In Model B, Latino pupils per- 
formed better at year end than had pupils observed in the other two 
groups, but they showed little growth from midyear to year end and did 
not meet the criterion. 

In addition to microobjectives specified for objectives in 
the proposal, the second-grade log had a collection of microobjectives 
for review of concepts called the Oral Skills Revitiw.«, All microobjectives 
of this review were like the first-language objectives of earlier grade 
levels. Results for these skills was uniformly high (close to 90%) each 
time they were observed. 
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Third Oracle 



During 1971-1972 , the third-grade . component v/as operational for 
the first time, in the Model A component " only . In addition to assessment 
of microofo jectives , the third-grade log of pupil behaviors contained a 
special test of two reading objectives, 6.1 (reading in the mother tongue) 
and 6.2 (reading in the second language). According to the proposal," at 
year end, 50% of the pupils should be reading in their first*" language at 
Level 3, Part 1, of their readers; 80% should be able to read in Level 
2; and all pupils should be able to read Level 1 successfully. Figure 
5.2 shews the performance of Anglo pupils. As can be seen from the 
figure, the pupils sairpled at year end were reading close to the distri- 
bution specified by the objective, with all pupils able to read at Level 

1 and into Level 2. Eighty percent of the pupils had been able to read 
to the midpoint of Level 2, and 50% of the pupils v/ere reading near the 
midpoint of Level 3. 

Third-grade Latino pupils reading in their mother tongue did 
not have the opportunity to read .at so high a level, as (according to 
supervisors) the most difficult text in use in the program was a Level 

2 text. As shown in Figure 5.3 at year end only 10% of the pupils had 
completed the second-year text (Level 2.2) , and the typical pupil was 
reading at the first half of Level 1 at both midyear and year end. This 
lack of growth from midyear to year end may point to problems of program 
mcinagement, especially the absence of Level 3 texts. 

Results for second-language reading showed that both Anglo 
and Latino pupils ware below stated objectives. The objective stated 
tliat all Anglos would be able to read the primer level and 40% would 
be able to read Level 1. As /:hown in Figure 5.4, by year end only 66% 
of the Anglos tested could successfully read a portion of the preprimer 
and, of course, fewer pupils could read the primer. No Anglos could- 
read Level 1. " 

The objective stated that all Latinos would be able to read 
Level 1, half would be able to read Level 2, and 10% would be able to read 
Level 3. Results shown in Figure 5.5 revealed that 80% of the Latino 
pupils tested could read the English preprimer at midyear, and some 
could read as far into the texts as Lev6l 2.2. Although this second- 
language performance of Latinos was less than anticipated, results were 
remarkably similar to those of Latinos in their mother tongue. Thus, 
the data seem to suggest that Anglo pupils in the program are . learning 
reading skills primarily in their mother tongue and only beginning to 
progress in their second language. In contrast, Latino pupils seem to 
be developing reading as quickly in English as in Spanish, their per- 
formance in the two languages being less than appropriate for their 
grade" level and less than stated in the objectives. 

Results for other objectives of the third grade Model A compo- 
nent are summarized in Table 5.8 and are presented graphically, by 
microobjective , in Figure 5.5. As can be seen in the table, only two 
of the. remaining objectives carried criteria — both groups of pupils 
were expected to master 75% of the skills designated for the second- 
language objectives (6.7 and 6.8) . Neither Anglo nor Latino pupils 
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Midyear - first language 
Year End - first language 



lOOu 



No. of Pupils Tested 

Midyear = 10 
Year End = 10 



Average Performance 

Midyear =. 22 (Pt.l) 
Year End = 3^ (Pt^i) 
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pt.l pt.2 pt,l pt.2 



pt.l it. 2 pt.l pt.2 pt.l pt.2 pt,l pt,2 

Reading Level 



Figure 5.2 Percent of 3rd grade Anglo pupils tested who could read the Bank 
Q Street reading texts to the criterion level. 
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Midyear - first language 
Year End - first language 
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Reading Level 

Figure 5.3 Percent of 3rd grade Latino pupils . tested who could read the 
Laidlaw reading texts to the criterion level. 



ERLC 



95 



— Midyear - second language 
year End - second language 



No. of Pupils Tested 

Midyear = 10 
Year End = 10 
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Average Performance 

Midyear = PP . (Pt. 1) 
Year End = PP (pt. 1) 
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Figure 5.4 Percent of 3rd grade Anglo pupils tested who could read the 
Laidlaw reading texts to the criterion level. 
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Midyear - second language 

Year End «- second language 
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Midyear = 10 
Year End = 13 
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Midyear = P^^ (Pt.2) 
Year End = P^ (Pt.l) 
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Reading Level 

Figure 5.5 Percent of 3rd grade Latino pupils tested who could read the Bank 
Street reading texts to the criterion level. 
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Table 5.8. Mean Percentage of Model A Third Graders Succeeding on the 
Model School Objectives. 



Objective 



Anglo 
Midye ar Year End 
N = 10 N = 9 



Latino 
Midyear Year End 
N = 10 N =14 



6.6 Spanish as a Second 
. Language (20 micro- 
objectives) Criterion : 
75% 



46% 



57% 



6.7 English as a Second 
Language / Latinos 
(33 microob jectives) 
Criterion: 75% 



72% 



6.8 Social Studies in 
First Language (16 
mi cr o ob j e c t i vo s ) 
Criterion: None 



71^ 



70% 



57% 



57%- 



6.9 Second-Language Social 
Studies (3 micro- 
objectives) 
Criterion: None 



63% 



11% 



4.3% 



55% 



6.11 Arithmetic in First 
Language (28 micro- 
objectives) 
Criterion: None 



63% 



71% 



51% 



83% 



6.12 Arithmetic in Secx>nd 
Language (8 micro-' 
objectives) 
Criterion: None 



59% 



58% 



45% 



63% 



succeeded in meeting these criteria, although by year end, the Latino 
perfonr.ance in English as a Second Language was very close to the anti- 
cipated level. In social studies/ two objectives which had no criterion werri 
designated, one for performance in the first language (6.8) and one for 
performance in the second language (6.9) . In the first language, pupil 
performance was the same at midyear .as at year end, with the Anglos 
at about 70%, Latinos at 57%. In the second language, Anglo perfor- 
mance was quite high (63%) at midyear but regressed sharply to 11% in 
the sample observed by year end. In contrast, Latino performance in 
social studies in the second language showed continued growth from the 
beginning to the end of the school year.. 
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The arithmetic objectives for first and second languages also 
lacked criteria for the perfomance of microob jectives in the logs (although 
criteria for the testing situation i"eported in Srudy 8 were " included) . 
In the first language some growth was observed in the number skills from 
midyear to year end for both ethnic groups, Anglos rising from 63% to 71% 
mastery, Latinos rising from 51% to 83%. In second-language arithmetic 
skills Anglo pupils* performances apparently remained t]*e same (at just 
lander 60%) , but Latino pupils showed growth from below the Anglo level 
(45%) at midyear to slightly above it (63%) at year end. 

Validity Check 

To assure that the data gathered by teachers were accurate, 
random samples of pupils in the log study were drawn and rechecked on six 
microobjectives . At midyear and year end 25 pupils were observed. Over- 
all, the results showed that there was 75,5% agreement at midyear and 
75% at year end, with a trend for the agreement to be slightly better 
in Model B than in Model A in both instances (71% versus 75% at midyear, 
77% versus 80% at year end) , At midyear the number of observations in 
which the pupils performed better in the validity check than they had when 
tested by the te^acher was nearly the same as the number of observations 
in which they performed more poorly. At year end the validity check 
showed that teachers reported more success than the evaluator was able 
to verify in three of four cases where discrepancies existed. 

Discussion 

Interpretation of the. outcomes of single objectives in this 
study must be done with great caution, because the number of observations 
which went into each is small- However, when trends emerge which cut 
across groups observation times, and grade levels^ these seem worthy 
of attention. 

One such tre.nd in the results of the 1971-1972 log study is a 
lack of visible growth from midyear to year end in most grade levels 
and language groups • This lack of growth seems to have two different 
causes. In the upper grade levels, little progress was made during 
the course of the year and the criterion was never met. This trend 
seemed to be stronger in the older, Model A program, suggesting that 
with familiarity with the operation of the program, the supervisors 
exerted less control to assure that classroom activities were consis- 
tent with the objectives of the program. For Model B this lack of 
growth from midyear to year end was less pronounced, possibly due to 
the use of the Miami curriculum materials which provided a greater 
amount of structure than was available with the materials used in the 
Model A component. 

In the lowest grade levels, the lack of visible progress from 
midyear to year end resulted from attainment- of nearly all objectives 
during the first part of the year. This pattern suggests that the 
stated objectives in the program may have been too easy for the pupil 
groups. Informal observation and discussion with teachers showed 
that most went beyond the written program guidelines in their teaching • 
Any enrichment of the microobjectives for the early levels could 
capitalize on the insights gained by these teachers • 
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SPECIAL FIRST GR/i.DE LOG 

1. How old are you? (Complete sentence), 

2. Where do you live? 

3. The child identifies these parts of his body: 
nose 

4. teeth * 

5. fingers 

6. feet 
1. face 

8. arms 

9. legs 

10, The child will respond to questions concerning the members of his family. 

What is your mother^ s name? • * 

11» What is your father's name? 

12. What is your sister's/brother's name? 

13. Shown a picture of the family unit, he identifies and talks about the 
members; mother, father, grandmother, grandfather. 

14. The child will. respond to questions concerning his school. 
What school are you in? 

15. What grade are you in? 

16. What is your teacher's name? 

17. The child identifies by means of pictures the following community helpers 
and describes their roles. 

milkman 

18. * garbage collector , - . 

19. newspaper boy 

20. Identifies: 

brown 

21. pink 
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22* Identifies: 

gold 
23. silver 

24* Hie child responds to the following commands: 
Stand up. 

25. Sit down. * . 

26. Show me, . 

27. Get the . 

28. Run. ' 

29. Skip. 

30. Jiimp. 

31. Choosing word pairs that begin with the same letter: 

four - five mama - mono 

four - six mama - libro 

32. Hie child talks about a situation picture. 

33. Re recites at least 75% of three poems. 

34. The child writes his own name. 

35. He counts from 1-100. 

36. When shown the following coins, the child will identify them: 

nickel 

37. dime 

38. quarter 

39. Identifies pint when shown containers. 

quart 

40. Identifies circle ^ ^ 
* triangle 

square 

y • 

41. Matches children to chairs 

children to pencils 

42. Using a clock, the child indicates an hour 

' ■ half>>hour • 

43» The child -tells tiir.e when shown a specific hour. e«g., 9'o*clock. 



44. When shown a picture the child is able to identify different -.inds of v.»ocither- 

' e.g., snowstorm 

\ , raining 

" ' . sunny • 

45. The child will place the appropriate weather symbol on weather chart. 

46* The child will respond to a question doncerning the weather. It is cold/ 

' ' . . hot/warm? 

47, The child melts an ice cube and states that it goes from a solid to a liquid* 

48, Identifies: turtle 

49, ■ frog* ■ 
50 * .9'oat 

51. ■ duck 

■52* ■ bear 
53. gorilla 
54* spinach 

55. cherries 

56. daisies 

57. When shown a picture of a furnished house, the child identifies the following: 





kitchen 


58. 


stove 


59> 


refrigerator 


60. 


mixer 


61, . 


sink . 


62. 


toaster 


63.' 


living room 


64.. 


sofa 


65. 


TV 


66.- — 


.lamp 


67.; ■. 


rug 


6^. 


vacuum cleaner 
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69. Identifies: dining room 

70. ■ . table 

71. buffet 

72. china closet 
73. . bedroom 

74. . bed . ' 

75. ' bureau - 

76. . ■ chair 

77. bathroom ■ 

78. bathtub 

79. toilet . 

80. washbasin 

61. vJhen shown pictures of community helpers and means of transportation the 
child will match them: 

e.g., fireman - fire engine 

82. milkman - milk truck 

83. policeman - police car ' 

Second Language 

1. -Where do you live? (whole sentence) . 

"2. shown a picture of a family unit he identifes and. talks about the. members. 

3. Waen asked the comJnand,^ Show me (your head), the child identifies the 

following parts of his body: 
head 

4* eyes 
5. nose 

.6. . ears ' ' 

7. mouth 

8. . ■ hands 

9. feet 
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10. Identifies the following family irrembers: 

mother 

11. father ' , 

12. sister/brother 

13. grandmother 

If 

14. grandfather 

15. Identifies the following coritPnunity helpers: 

milkman 
16 • policeman 

17. fireman 

18. Shown a color chart, the child identifi.:is these colors: 

brown 
pink 
gold 
silver 

19. The child recites at least 75% of at least one memorized poem. 
20* The child recites at least 75% of a second poem. 

21, The child recites at least 75% of third poem. 
22^ Ihe child responds to the following commands. 
Sit down. 

23. Stand up. . 

24. Give me. 

25. When told to find something wxth a certain shape, the child is able to 
point to: a circle 

26. a triangle 

27. a rectangle 

28. a square 

29. The child will respond to a question concerning the weather. It is cold/hot, 
warm, cool, etc. 

30. When shown pictures the child identifies the following animals- 

cow 

31. chicken 
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32. When shown pictures the cMlH irie^n^^ -p-i *.u^ ir - t • 

f wici, L.iti u.*i±a laentiries the foxlowing animals: 

hoise 

33. turtle 

34. frog 

35. goat 

36. duck 

37. Identifies: lettuce 

spinach 

cherries 

daisies 

41. Wlien shown a picture of a furnished house, the child identifies the following: 

kitchen ' 
^2. living room 

dining room 

44. bedroom 

45. bathroom 
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Reading in the Mother Tongu e; 

a. Ask pupil to read aloud the first 10 sentences of the last page he has 
read in class from his reader. 

b. If he reads the sentences ^p§rfectly or with less thc-^n four errors^ go 
forward five pages, and ask him to repeat the. procedure, 

c. If he makes five 02; more' errors, go back in the boOK'- f ive pages and 
ask him to read again. 

d. Repeat b or £ as needed (until you find the most diffic\ilt place at 
which the pupil can read 10 consecutive sentences with less than- four 
errors) but not more than three times. if, after this third repetition, 
the place is not found, stop the testing, if the pupil is at the 
beginning or end of a reader, it may be necessary to change books to 
complete -the process. 

1. place where pupil is reading in class. Book- - . , Page 

Number of errors made_ 

2. First 'R^ijetitiOii: SooX '_■ . - , Pcage . ■ -> ^ iNo. of errors 

3. second Repetition: Book " page , No* of errors 

4. Third Repetition: Book . ; . ■•- . . - y. page ' , No, o£ errors 

Second Language 

Repeat the process described in Item 1 above using the reader the pupil 
is using for his second language, except that the pupil need, read only five, 
sentences of each sample. Record the results below: 

1, place v/nere pupil is reading in class. - 

Book / Page , No*, of errors 



2. First Repetition: Book " ' ^ page , .No. of errors 

3. Second Repetition: Book • ' . \ . ?age^^ , No. df errors 

4. Third. Repetition: Book ■ , ?age_ , No. of errors 



Note If the pupil is reading the Bank Street pre-primers have him.. start at . 
the beginning of his book and continue reading until the end*. Indicate be lov/ 
the number of errors made on each page. If the pupil can not read -further-,, • • 
indicate the point where the testing^ was stopped*. Book . " . ' ■ ' / • 

Page , No. of errors 



Page . , No-, of errors 
JPage . , No. of errors^ 



Page / No* of errors_ 

page_ ^ r No. of errors 

Page No. of errors 



3. 
4. 
5, 



Count by 'S's to 50. (Level 5) 

5 + 8 = /--(Level 5) 

Pick the one whj.ch shows 1/4 of a circle. 



- i 



6. 
7, 



How many 10^s <r,e .in 32. (Level 5) 

How many is 5 (Level 5) • 

2 



(Level 5) 



8» How many qi>^^ ' iij in one dollar? 
. ■ or ■ 

dcuantas pesetas hay en un peso? (Level 5). 

9. Ask pupil the nu'-nber v/hich ccmes before 20. 

10^ 2x4=. (Level 5) \ ^ 



(Level 5) 



.11. 



,/ 



Hov; m\3ch of the circle is' dark? 1/3' h ■ (Level 6) 



/ 



12. Read the following aloud - 253 (Level 6) 

13. Read the following aloud - 845 (Level 

14. Read the following aloud - $2.60 (Level G) 

15. Hov^ ma-ny ones, tens . and hundreds in 125? (Level 6) 

16. 18 (Level 6) 

• -2 ' ■•■ 



17. 31 (Level 6) 
{IQ* 2 flT (Level 6) 
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1\'K Wliat nuiabci: is 2,035? Cevcl 7) - 

20. What number comes- before 425? . (Level 7) 

21. Add 22 (Level 7) 

+36 ■ 

■ 25 ■ 

22. Add 2.43 (Level 7) 

+ 1,51 

23. . 9x8 = (Level 7) 

24. !■ 12 (Level 7) 
• X 2 ^ 

25. 3/64* (Level 7) ^ ^ 

26. Count backwards from 50 to 10 by lO^s. 

27. Read the following Roman Number IX. 

28. Read tlie following Roman Number VI. 

29. COT.pute • (2::10) ; 5x1) == 

(accept either. 20+5) or 25. 

30. 400 

>^126 

ARITKMETICf Second Language 

31. count- by 2 ' s to 20. (Level 5) 

32. ' How many lO»s are in 24? (Level 5) ■. . 

33. How many quarters are in one dollar 
i'Cuantas pesetas hay en un peso? \Level 5) 

34. Read the following number aloud 253, .'(Level 6) 

35. Read this number aloud $3.80. (Level 7) 

36. V/hat number comes after 425? (Level 7) 

37. count backvjards from 50 to 10 by lO's. (Level 8) 

38. What- number is this VI? (Level 8) 



■ . ' ST.CO'^lj 7. 7;: ::-vnCK CtoD, Anrrlos zr)c[ Latinos ) . " 

39". Iclenl-.ii ios ■ f andiy rr-.ir^iri.ars (nouher; fachor, sori/ ciautjhter) f:rom a picture. 
40. Responds to "Sr.ov; 'f.H- ^/^^ur ears!'* 

41^ Re.'iponds to r'TJhow mo youi* foot i " . . • . 

42. Identifies the following cojrartunity helper (from picture) mil/jman# 
.43. Identifies the follov;ing community helx>er (from picture) policeman. 

44. .Follov/s the following comrr^and. Raise your hand! ' • 

45. Follows the following comniand. Sit down] 

46. Child points to object in the shape of a square. 

* 47. Shown a picture, the child ' identifies a horse. ' ^ 

48. £hov;n a picture, the child identifies apples. 

SECOND LANGUAGE Spanish .(Anglos only) 

49s Give the child a pencil and say -dTienes un libro? Correct" answer: 
NOr no tengo un libro » 

50s d Que tienes? (Answer Tengo uri lapiz.) 

51s d Que dia es hoy? or dCual es la fecha? .. 

52s Give child five pieces of color paper (red,- black, blue, green, white) and. 
ask. - -^Cuantos papeles tienes? (Answer: Cinco) 

53s c De que colores son los papeles? (Rojo, azul , • ve'rde , negr^o, bianco) .. 

54s Shov: child clock or pictures of clocks reading 0:00 and 3:00. Ask - 
(JQue hora es? Answer: Son las tres, son las nueve. 

55s ci Cuantas manos tienes? 

■ 5&J Cuantos dedos tienes? . . . 

57s c^Cuantos ojos tienes? 

5Ss Shown a picture oi ^ i^'^'uny scene r the teacher says - 
J Que ticmpo Ivrrcr llc^cr- - r=.o!l.,' cialor, buen tiempo. 

SECOND Language English (Latinos only) 

49e '.The pupil is given a pencil , and teacher says ''Do you have a book?" 
Answer: No. 

50e What do you have? ' ' . • J 

Answer": pencil. . . • ' * . 



51e "How' many pieces of paper do you have?" 
52e "v*iat colors are the papers?" 



i/3.c Hav; r.^-'iv oyc r c:/ you Iju z-jV 
S4e Hov,' rrj'isny : cvr. r; j you -jrrv'cy 

55e Jfov; rnsny no:-;or. clo you ruivo? 
56e How mr*ny f iac;crs clo youV 

57e Shown a picture of a sunny scene, the teacher askS: ^'K'hdt's the weather?" 
Answer; Ic'c v/arm, hot;, sunny. ^ 

58e Shovm a picture of clocKs showir^g 3:00 or 9:00 o'clock, teacher says n 
"Ivhat time is it?" 7-^nswer: Itfs 3^ o'clock, It!> 9_ o^clock. 

Kitchen Show pupils the "kitcheri' picture . 

59e vtnat room is this? 

60e Who's in the picture? 

6ie VJhat is she doing? 

62 e V^at do we do in the kitchen? 

63e v/hat do you see in the picture? 

stove - . 

pot ^ ■ . • 

table 



ERLC 



tablecloth 

cloth 

sink 

clock 



refrigerator 



64e Chile speaks freely. 

Child needs prorr.pting, 
(check one) 



65e child speaks in a sentence. 



Child speaks in isolated words. 

(check one) 

Story The Story of Ferdinand 

(Play record up to "So they took hijTi away for the bullfight in a cart.'') 

66 » What was the name of the bull? 
(Ferdinand) 

67, \\/hat did Ferdinand like to do? 

(He liked to sit quietly and smell the flowers) 
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(They iil-.-.-.C: to run, una iun.p, unc^ fri-'jht). 

{'iif^ Sci'^ or) a buii;hIel/^-f; ) • 

70. v;.iO did mc-n Xvick to fj'j/jt in the? bulliit^rit? 

71. VVhoL do you think hapx^cned to Kcrdinandv (/.ccopt any tinding the child 

ciives, ci:j long as it is clo.ar that ]\e understood the story up to the 
point where the record wais stoiDped) . 

SOCIAL STj:)rr:Sr First T^gjUfuano 

72. VJliat is the difference between ■ clothes we wear in v/intcr and in the sujnmer 

in Philadelphia? (VJeather) 

73. What kinds of weather do we have in winter but not in summer? (In Phila#) 

Answer: snow, hail, etc.. (Weather) 

74. • \^y don't v;e go to the shore in the winter? (Weather) 

75. what animals give us v/ool? Answer: sheep (Clothing) 

76. \/hat is the difference between party clothes and school clothes? Answer:. 

party clothes are fancier, etc. (Clothing) 

77* >Iow is iTianufactured clothing Drought to our stores? Answer; trucks, trains. 
(Clothing) 

78. IVhat machines or appliances do people have in their horaes? Answer: washing 
machine, vacuum cleaner, TV, radio, etc. (Machines) 

79* .^;ame some things v;ith motors in them. (Machines) 

80. Name some things we use that are found in the earth # Answer: iron, coal, 

diamonds, etc. (Earth treasures) 

81. Vv'hat are sor:e things m:-de of iron? Answer: pans, cars, etc. (Earth treasures) 

82. Vvhat are some foods that grow underground. Answer: potatoes, carrots, peanuts. 

(Earth treasures) 

83. Which foods that we eat come from the sea? Answer: fish, clams, shellfish, 

etc. (Sea) 

84. Why do some foods come in cans or come frozen? Answer; To >keep them fresh. 

(Foods ) . ■ 

85. Name some foods made from milk? Answer: cheese, bufcer, etc. (Foods) 

86. What do plants need to grow? Answer: sun, water ^ soil (Plants) 

87. ;^y are moLst trees green in Puerto Rico, but lose their leaves in Philadelphia? 

Answer: Clinate difference (Plants) 
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ii-:.:z-y.~i T-f' KGuAC:?;, Social S 1: :Ics . . 

88. ^/hot kinds of weather do we have in winter, but not in sunmer in Phiiav ^Wearhor) 
fiy. I'fiiat anijnal yives ua v;oolv (Clothincj) 

90. VJhut foods that we eat come from the sea? (Sea) ' 



I 

} 




ERIC 



125 



STUDY 6. PPSKIMDSRGARTEU FuPILS' READI^]£55 TOR AN ALL-DAY KINDi:.HGARTKN 
PR0GRAx*4 IN^ MODEL A 

The Prekindergarten Readiness Test is an instrument designed to 
assess the effectiveness of the prekinde^rgarten level of the Model. A school 
program in readying pupils fnr an enriched kindergarten program. The test 
measures reading- and number-readiness skills. Some of these skills related 
to objectives which appeared in the proposal for the first year of the 
program: 

Objective 1,1 (First Language) included microobjectives related 
to color identification. 

Objective 1^2 (Number Skills) included shape discrimination and 
counting activities, 

In addition to these, the prekindergarten test includes letter- 
discrimination activities added to the program after the objectives were 
written. 



Procedures 

Program 

No major changes were implemented in the prekindergarten program 
in the 1971-1972 school year. Pupils were again taught by a team of 
teachers, one Anglo and one Latino. Pupils were instructed in their mother 
tongue by the teacher who was the native speaker of the language, but 
received second-language instruction from the other teacher. The pupils 
worked in ethnically homogeneous groups for part of the day and ethnically 
mixed groups for the remainder. Teaching focused on oral skills, reading- 
and number-readiness concepts, and social and emotional development. The 
program included activities which were similar to those required by all 
test items; however, direct practice with the test was avoided. 

Evaluation ' 

Sample . All pupils on roll in the Model A prekindergarten were 
included: 37 Anglos and 34 Latinos. 

Instrument . The Prekindergarten F^adiness Test was described in 
detail in thr^ first year's evaluation report [Offenberg, 1970). It contains 
items which require discrimination of shapes and letters, identification of 
colors/ use of simple number concepts, and ability -o make marks with a 
crayon. 

Administration . Each pupil was tested individually by a super- 
visor in the program* The testing was conducted in the pupils' mother 
tongues during May, 1972. 

Analysis . As this instrument's properties have not been studied 
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in previous evaluations, an item analysis was conducted- The mean score 
for each of the ethnic groups was computed, and a comparison of this year*s 
jDupil performance with last*year*s was made by means of an analysis of 
variance. The 1970 data were excluded from the variance analysis because 
individual scores for that year were no longer available. 



Results 

Item Analysis 

Results of the item analysis of the test are shown in Table 6,1. 
The analysis was based on the results of testing of 34 Latinos and 19 Anglos* 
The remaining Anglo pupils' had to be excluded because one of the testers 
neglected to mark the pupil responses to the individual items and indicated 
only a total score. 

As can be seen on the table, the first six items ^ dealing with 
shape and letter discrimination, were mastered by virtually all pupils 
(94.5% to 98.2%), with a fairly strong tendency for pupils who do well 
on these items to do well. on the test as a whole. Correlations of the 
items with total score were between .57 and .59, all statistically signifi- 
cant at the .01 level of confidcince. 

The number-concept and numerical operation items (numbers 7 
through 17) were more difficult, with about half the pupils succeeding on 
the typical item. All items correlated significantly (at least at the .05 
level) with the total score, with correlation coeff icients . ranging from 
.30 to . 57. 

The color-discrimination items were mastej^d by most pupils, but 
the overall results did not correlate so highly with total score as did 
other items. Between 80% and 94% of the pupils completed each item 
correctly, but the correlations with total score ranged between .24 and 
.38. All but one of these correlations were statistically significant 
and that one just missed being significant. Kuder-Richardson 20 reliability 
was good, and other test properties were approximately at the anticipated 
levels. 

Pupil Performance . Overall performance of pupils on the Prekinder- 
garten Readiness Test was very similar to the performance of pupils the 
previous year, with the typical pupil scoring 17.5 items or 76% of the test 
correct. As shown in Table 6.2, there were small differences in the scoring 
of the two ethnic groups last year and this_year, but the overall results 
were virtually identical. Neither group's performance was significantly 
different from that observed last year. 



Discussion 

The item analysis shows that the prekindergarten test reflects 
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TABLE 6.1 



ANALYSIS OF PREKINDERGARTEN READINESS TEST ITEMS, 
SHOWING CORRELATION OF ITEMS WITH TOTAL SCORE 



Item 



Percentage 
of Pupils 
Succeeding 



Point-Bis erial 
Correlation 



7orm Discrimination 
(Correct answer underlined) 

1. F K E B 

2. A O ^ Q 

3. O O 0 A 

4. 3 1 _2 4 

5. 12^2 
5. B C A E 

Make nuinber of marks shown 

7. 2 

8. 3 

9. 5 
10. 6 

Find the box with four dots 
11* 

How many dots in the circle? 
12. 

12 3 



13. 



14. 



15. 



9 6 5 



5 4 6 



2 3 4 



94.5 
96.4 
96.4 
98.2 
94.5 
94.5 

94.5 
•7 8.2 
49.1 
36.4 

50.9 

81.8 
to. 9 
49.1 
54.5 



.51** 
.55** 
.58** 
.59** 
.51** 
.51** 

.57** 
.46** 
.47** 
.38** 

.51** 

.30* 
.40** 
.44** 
.31* 
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TABLE 6,1 (Continued) 



ANALYSIS OF PPEKINDERGARTEN READINESS TEST ITEMS, 
SHOWING CORRELATION OP ITEMS WITH TOTAL SCORE 



Item 



Percentage 
of' Pupils 
Succeeding 



Point-Bis erial 
Correlation 



How 


many dots in the circle? 






16/ 


5 6 


56,4 ^ 


.45** 


17. 


9\ 4 £ • 


56:4 


. ',36** . 


Show the color indicated 






18. 


Blue Orange Green 


88.1 


,27* 


19. 


Purple Yellow Red 


94,5 


5 38** • ■ 


20, 


. Blue Red Yellow 


90.9 


,24 


21 


Green Yellow Red 


80.0 


.38** 


22. 


Blue . Red Orange 


92.7 


.34** • 


23, 


Blue Orange Purple 


83,6 


-.30* 




Summary 






Number of items on the test 




23 




Mean number of pupils succeeding 
on each test item 




; 76.4 % 




Kuder^^Richardson 20 Reliability 




,84 • 




Test Mean ' 




17.78 




Variance 




17.06 




Standard Deviation 




4.13 




Standard Error of Measurement ' 




• . 1.61 



. *Statis"tically significant at: the .05 level, 
T-r)^^' **Statistically significant at -che ,01 level. 



^ ■ TABLE 6.2 . 

• COMPARISON- OF SCORES ON PREKINDERGARTEN READINESS TEST 



Year Anglo " 
N X 


Latino 
N x" . 


Total 

. N " X 


1972 - 37 


16.8 


34 IS. 2 


71 17.5 


1971 ■ 33 


■ 18.3 


37 17.0 


70 17.7 




r\l 1 CI ^ y «D X s 


of Variance 




Item 


MS 


df F 




Anglo Pupils 








Treatment 


39.0 


1- 1.55 


ws 


Error 


25.2 


68 




Latino Pupils 








Treatment 


21.3 


1 1.16 


NS ' 


-Error 


18.3 


69 
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criterion- referent approach, in that most items were completed correctly by . 
more than half the pupils and many items were completed correctly by more 
than 80% of the children. The most difficult items were clearly those in 
the number iskill area . it is not known whether the difference in achieve- 
ment on reading readiness items and on number readiness items was a re- 
flection of the tests administered or of th/. teaching procedures in the 
tv;o areas* It does suggest that examination of the teaching procedures used 
for the number concepts may be desirable. 

The overall pupil performance suggests that the project management 
can expect about the same number of pupils to be tceady for the all-day 
kindergarten experience in 1972-1973 as there were in 1971-1972. 
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r imEhVjhR'VEU Pb-pn.rs* kKAniNKSS FOR TJiK FIRST GRAOPJ, INCLUDING 
A MEASURE OF TUE IMPACT OF THE ALL-DAY KINDERGARTciN 



The Philadelphia Readiness Test has been used as a criterion 
measure for assessing reading-readiness skills and number-readiness skills 
in all three years of the Bilingual Program's operation (Offenberg, 1970, 
1972) . Performance Objective 2.7 in the original proposal stated that both 
Anglo and Latino pupils' Performances would exceed that of two critical groups — 
all pupils in the city in the Spring 1969 testing, and pupils enrolled in the 
schools which merged to become Potter-Thomas in the same testing. The former 
group had a mean score of 20*1; the latter group, 20.9. 

During the first year of operation, both Anglo pupils tested in 
English and Latino pupils tested in Spanish met this objective, the Latino 
pupils scoring higher than had pupils in any school in the city during the 
base-line year. During the second operational year, results were mixed in 
the Model A program, with Anglo pupils achieving their objective, but with 
Latino- pupils scoring below both criteria. During that second operational 
year, a kindergarten class, consisting solely of Spanish-speaking children, 
was established as part of the Model B component. The performance of this 
group was well above the criterion for the objective. An all-day kindergarten 
was also begun in Model A which consisted of select pupils with prekindergarten 
experience. This group performed well above the objective's criterion. 

During the third operational year of the program, all . classes 
operating in the Model B component were at higher grades than the kindergarten, 
but the regular and all-day kindergartens were continued in Model A. 
Therefore, this study focuses on the Model A component alone. 

In addition to assessing the effectiveness of the program in teach- 
ing pupils the reading- and number-readiness skills designated in the objectives, 
this study undertakes an assessment of the effectiveness of the all-day kinder- 
garten program in speeding up the skills acquisition of the brightest alumni 
of the prekindergarten component. 

Objective 

Product objective 2.7 stated that the mean scores of both Anglo and 
Latino pupils in the kindergarten bilingual classes would exceed the citywide 
mean of 20.1 and at least equal the Potter-Thomas School mean .of 20,9. These 
means were for the base-line year of 1969, the year before the Bilingual 
Program began. 

Procedures 

Program 

According to the proposal/ all pupils were to be given experiences 
in which they would have opportunity to practice letter and shape discrimina- 
tion and practice elementary numerical operations. The products of .these 
activities were described in detail, but the specific methods used for 
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instruction were not. According. to the supervisors, teachers did not sub- 
stantially alter the methods used to teach the number- and reading-readiness 
skills from those of the preceeding year. In the regular kindergarten, 
emphasis was on readiness skills, in the all-day kindergarten, emphasis was 
on readiness and beginning reading skills. The specific microobjectives 
around which the teachers built their classroom activities are detailed in 
Study 5. One team of teachers (regular kindergarten) supplemented the regular 
materials drawn from the Miami Curricumun Development Center, also described 
in that study. 

Evaluation 

Instrument . The Philadelphia Readiness Test and the Spanish directions 
prepared for it were described in detail in the first yearns evaluation report 
(Offenberg, 1970).' The test requires pupils to copy shapes, match letters 
and words, and perform simple counting and numerical operations up to ten. There 
is not a one-to-one correspondence between test items and program microobjectives, 
but each item can be viewed as a sample or an extension of an objective in the 
program. 

Instructions for the test, which has been used extensively in the 
Philadelphia schools, were originally in English, For project purposes, 
a Spanish translation of the test in£:.tructions was prepared. In the first 
year of program operation, Latino pupils were randomly assigned to groups who 
were tested using the English and Spanish instructions. It was found at 
that time that the Spanish instructions enhanced the performance of Latino 
pupils. Since that time, Anglo pupiJ s have been tested in English, and 
Latino pupils have been tested in Sprmish. 

Population and Administration , During the 1971-1972 school year, 
all pupils enrolled in bilingual kindergarten x^^rograms were part of the Model 
A component at the Potter-Thomas School. Seventy-seven Anglos and 83 Latinos 
were enrolled in the regular kindergarten classes; 20 Anglos and 21 Latinos 
were enrolled in all-day kindergarten programs. According to teachers, this 
group included all children on roll during the April 15-May 15 testing period. 
All but 14 of these children were on roll prior to December, 1971. As the 
scores of these children were not different from those of pupils who had been 
in the school the entire year, both groups were combined in all the group 
statistics presented. 

The population of the kindergarten component consists of pupils 
enrolled in the program by their parents. As kindergarten is not mandatory 
in the Philadelphia schools, this group may to some extent be regarded as 
atypical: parents had to choose that their children be enrolled. However, 
the same atypicality also affects the base-line groups, as base-line children 
were enrolled in kindergartens prior to program implementation under the , 
same conditions. 
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The a 11 -day kinrlerqarterv croup is more select. It consists 
primarily of children who v/erci successful in the prekindergarten progrcim . 
during .1^70-1971. They v/ere assigned to classes on the basis of high scores 
on the pr;ek.indGrcarten torjts administered at the end of that school year. 
Thye were in about the top two-thirds on that test and. were recommended by 
their teacher. A few talented pupils who had not prekindergarten experience 
were also assigned to' the all-day kindergarten on the basis of teacher 
recommendation. 

Tests were administered by teachers in small groups of five or six 
pupils /. usually in two sittings. Each pupil's test was administered in the 
pupil's mother tongue. The supervisors reported that they spot-checked teachers 
administering the tests and they appeared to be following the test instructions. 
Teachers subsequently scored the tests. The evaluation staff reviewed the 
tests and the scoring (a) to assure that the tests appeared to be the work of 
the- pupils themselves and (b) to check the scoring. Three tests were rescored 
after it was observed that the teacher had made an error. 

Analysis . Descriptive statistics (mean and standard deviations) 
were obtained to determine whether the objective was met. Analysis of , 
covariance was used to determine whether the all-day kindergarten enhanced 
participant performance to levels beyond that which could be anticipated by 
the pupils' ability, in this analysis, the previous year's prekindergarten 
test score was used as a covariate. 

Results 

As shown in Table 7.1, both Anglo and Latino pupils in the program 
exceeded the level of performance specified in the objective, with mean 
scores of 23.2 and 22.7 respectively. A mean score of 20.9 was necessary 
to exceed the objective's criterion. As anticipated, both Anglo and Latino 
all-day kindergarten pupils scored higher than children in the regular 
kindergarten, Anglos scoring 24.5 and Latino scoring 25.3.; These latter" 
scores are close to the ceiling of the test. This and" the fact that the 
standard deviations were- smaller for these 'groups than they were for the 
"regular" groups suggested that the degree of difference between the regular 
and all-say kindergarten groups might have been underestimated by the lack 
of more complex- test items. Check of the individual . test scores suggests that 
this was so: 19 of - the 41 all-day kindergarten pupils either had perfect 
scores or made only one error. 

Table 7.2 shows an' analysis which attempted to examine whether the 
all-day kindergarten component enhanced scores on- the Philadelphia Readiness 
Test. As it was known that pupils in the two groups were different before 
they entered" kindergarten, an analysis of covariance. was computed for those 
children who had been in prekindergarten the previous year. This permitted 
the use of prekindergarten readiness test scores as the covariate. The 
results in table 7.2 show that there were no statistically significant . ' 
differences once the prekindergarten results were taken into account. This 
indicated that at least as far as the Philadelphia Readiness Test is concerned, 



Table .7.1 Performance of All Pupils in the Model A Component on the 
Philadelphia Readiness Test. 







Number of 


Mean 


Standard 


Group 


Pupils 


Score 


Deviation 


Anglo Pupils 








Regular Kindergarten 


83 


22.8* 


4,0 


• All Day Kindergarten 


20 


24.5* 


• 3.8 ■ 


Anglo Total 


.103 






Latino Pupils 








Regular Kindergarten 


7.7 " 


.22.1* 


■ 4.7 


All Day ' Kindergarten 


21 


25,3* 


2.1 . 


Latino Total 


98 


22.7* 


4.5 


Program Total 


201 


. 23.0* 


4.3 



*Indicates that group exceeded the criterion of both baselines* 
Maximum test score is 27; preprogram city mean was 20.1; preprogram Potter- 
Thomas mean was 20^9. 



the all-day kindergarten did not produce pupil performance beyond that pro- 
duced by the. regular kindergarten once pupil ability v;as taken into account. 
However, examination, of the group means, in Table 7.2 suggests that the lack 
of significant differences may be accounted for by the same lack of difficult 
items noted earlier, as the all-day kindergarten group mean was very close 
to the test maximum. 

■ Discussio'h " 

The results showed that reading- and number-readiness skills have 
progressed beyond the levels of performance mandated by Product .Objective 
2.7. Anglo performance was higher than it had been in any previous year. 
Latino performance was nearly, as good -as Anglo performance and approached the 
outstanding results obtained for this- group during the first year of program 
operation. 
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Table 7.2 Analysis of Difference in Performance of .Last Year's 

Prekindergartens in Regular and All-Day Kindergarten Using 
Spring 1971 Prekindergarten Readiness-Test Scores as Covariate. 



Group 


N 


PK Test 




Phi la. 


Readiness 


Test 


Anglo 














All-Day Kindergarten 


17 


21.9 






25.2 




Regular Kindergarten 


8 


14.4 






, 22.8 




Latino ■ 














All-Day Kindergarten 


17 


20.5 






25.6 




Regular Kindergarten 


9 


12.8 






23.1 




Analysis. 


of Covariance 








Item 




SS - 


.df 


Ms 


F 


P 


Test of Equality Regression 














Within cells 




231.20 


43 


5.38 






Regression 




14.05 


3 


4.68 


0.87 


<(. 46 


Analysis of Covariance 














Ethnic Group 




6.01 


1 


6.01 


1.1. 


NS 


Program 




' 2.05 


1 


2.05 


0.4 


NS 


Ethnic Group x Program 




0.02 


1 


0.02 


o.p 


■NS 


Within cells 




245.02 


46 


5*30 






Regression 




.30.08 


1 


30.08 


5.7 


^•02 



While there is no clear explanation for these improvements ^ project 
supervisors were able to offer some hypotheses. Two new'teachers (an Anglo 
working in the all-day kindergarten and a Latino working . in a regular . kindergarten 
class) replaced two members of last year's kindergarten staff. In addition to 
having taken part in the 1971 Slimmer Institute, the Spanish-speaking teacher 
had had experience in teaching in Cuba prior to coming to the United States. The 
Anglo teacher also, was experienced. Furthermore, two classes also began to use 
materials prepared by the. Miami Curriculum Development Centers as supplements . 
^ It is felt that these materials may have contributed to the' high level of per- 
formance observed. 



siqnificanrdi^^ already been noted that the failure to find statistically 
were\"eithJi fhr""'l'" P--f°— ce of prekindergarten aluznni who 
were in either the regular or the all-day kindergarten might be attributed 

1° ?hfalTdav" ^"^f^^-r^^-- should be noLd, however, that p^pi s 

''J^^J J'^^-^^y _^^^^^^<3arten group not only were exposed to readiness skills 

the\f re^sSt: s^t't^that^the^ljl^^/^v^" '^^'^ Taken^tog^tiir , 

While un^lertakin^ r that the all-day kindergarten program may be a worth- 

I^ihe observed L r"*'" " '"'^ °' statistically significant differences 
in tne oDserved performance of the pupils. 
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study 8. STANDARDIZED TESTING OF MODEL SCHOOL PUPILS 



Over the first two years of program operation, the major thrust of 
the evaluation effort has been to provide information which might be useful 
for further project development and improvement. In short, the evaluation 
approach was formative. In the third year it was felt that some summative 
evaluation, in the form of standardized testing and program base-line com- 
parisons should be included in the evaluation. For the Model School program, 
this approach meant administration of standardized tests for which there 
were meaningful base-lines: selected subtests of the Stanford Achievement 
Test Primary 3attery II, The Iowa Tests of Basic Skills, and the Test de 
Destrezas Basicas en Lectura. They were used to assess performance in the 
mother tongue of all second- and third-grade pupils, and second-language 
performance of third-grade Latinos in the Model School program components. 

During the second year of program operation, this kind of testing 
was begun when second-grade Anglo pupils were examined in English with the 
Stanford Achievement Test. This was undertaken in order to assure that 
participation in a bilingual classroom program was not interfering with this 
group's acquisition of English reading skills, with the results showing that 
Anglo second-grade pupils of Model A were ahead of similar pupils prior to 
the start of the bilingual program. 

Procedures 

Program 

All three tests to be used in this study measure reading skills in 
the pupil's mother tongue. The methods for teaching these skills in the 
Model A program are described in detail in the second-year evaluation report 
(Offenberg, 1972) and Study 9 of this report. It is sufficient to say that 
the Latino pupils in the program ■ shov; the cumulative results of their study of 
Laidlaw materials, and Anglo pupils show the cumulative result of their work 
with the Bank Street materials.. (The Lippincott reading series was used as 
a supplement to the Bank Street materials) . 

The Model B program for Latino second graders is s'obstantially the 
same a<s that used for the Model A program, but that of the Anglos is not. 
The Anglos' study Of most of their mother-tongue skills, including those measured 
by the Stanford Achievement test, were planned and taught independently of the 
control and supervision of the Let's Be Amiqos project management. The approach 
used was phonics-based and was developed by Behavioral Research Laboratories. 

Some comment is required about "the reading programs of the base-line 
groups with whom comparisons were made. The base-lines for English reading 
skills came from data gathered in the Potter-Thomas School in years before 
the program reached that grade level. That of the second-grade group consisted 
of pupils tested in 1970, during which year students were exposed to an eclectic 
reading program; i.e., teachers planned activities based on (a^) the pupils in 
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their classes and ih) the teachers* own preferences. The base-line of the 
third-grade group was gathered in 1971 after the first year of a local-district- 
wide program operated by Behavioral Research Laboratories, which emphasized 
Phonics and decoding skills and included activities (such as poster-making 
contests) which would generate enthusiasm among staff and pupils. 

The base-line group for data Ot'i the Spanish-language Test de Destrezas 
v/as obtained from a 1968 citywide testiuc/ program (Desing, 1968) in which all 
pupils in the second through eighth grades who, in their teachers *• judament vyere 
Spanish-speaking were examined in that language. At the time of this testing, 
the school system was not offering any instruction in reading in that language. 



Model A third-grade Latino pupils were thought to have a sufficiently 
good mastery of English language skills to be ready for a testing in that 
language witi a first-to-second-grade level test. The participants in this 
program had studied aural-oral English exclusively until the last half of the 
second grade or the beginning of the third grade. At that time reading was 
introduced using the Bank Street series of readers. 

Evaluation 

The procedures used in gathering the evaluation data for this study 
were similar regardless of the instrument or target group. 

Instruments . The Stanford Achievement Test Primary Battery II was 
used to examine the reaching skills of three groups: second-grade Anglos of 
the Model A component, second-grade Anglos of the Model B component, and third- 
grade Latinos of the Model A componei:t* Four subtests — Word Meaning, Sentence 
Meaning, Spelling, and Word Study Skills — were administered. 

The Iowa Tests of Basic Skills- -Vocabulary , Reading, Language, and • 
Arithmetic subtests and the Composite score — were used to assess the performance 
of third-grade Anglo pupils in their mother tongue (Model A only; Model B did 
not have third-grade classes in 1971-1972) . 

Latino second-grade (Model A and Model B) and third-grade^ (Model A) 
pupils were examined on a Spanish reading test, Test de Destrezas Basicas en 
Lectura. The instrument provides scores for Words and Letters (recognition) , 
Word Meaning, and Comprehension, and a Composite score. 

Method, All testing was conducted by classroom teachers. To assure 
correct, fair administration of the tests, supervisors visited the classrooms . 
during test administration in Model A, and a research staff member assisted 
the teachers in testing Model B. All tests were administered within the two- 
week period which ended in early May. This time period is used also for citywide. 
testing of pupils. 
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In all cases the testing was given according to test publishers' 
^instructions and scored in accordance with the keys the publishers provided. 

Subjects . All pupils in the classes which composed the second grade, 
of Model A and Model B ani the third grade of Model A were to be examined in 
this assessment. The number of pupils enrolled in each grade and the percentage 
of pupils tested (April, 1972) were as follows: 

Model A; Anglo second grade, 58 (67%) 

Anglo third grade, 58 (96%) 



Latino second grade, 83 (87%) 

Latino third grade, 99 (95%) 

Model B: Anglo second grade, 29 (93%) 

Latino second grade, 41 (95%) . 



The number of pupils who actually took each t^st or subtest are 
shown in the tables in the Results section of this report. The discrepancies 
are due to absenteeism. According to teachers, no pupils were excluded from 
the testing for other reasons. It was not clear, at the time of this writing, 
why only 67% of the r.econd-grade Anglos were tested. 

Analysi s . All analyses in this study were of the "analysis of 
variance" type. As the data analyses proceeded, the authors revised their 
idea of the best way to handle them. As a result, there are some, variations 
within the study; specifically, in later portions, multivariate analysis of 
variance was substituted for several univariate analyses. In these analyses, 
pupil performance was compared with that of base-line groups. There were all 
historical base-lines; i,e., they were for pupils who were similar in ethnic 
background and believed to be similar in social cla^s and school experience to 
those pupils served by the program. The specific groups used are shown in the 
Results section. 

Results . . 

Performance of Model A and Model B Anglo Second-Grade Pupils on the Stanford 
Achievement Test 

Group means, grade equivalents of the means, and standard deviations 
of the test scores of first-grade pupils on four siibtests of the Stanford 
Achievement Test are shown in Table 8.1, which also includes variance analyses 
of the outcome. The base-line group used in this analysis was all Anglo pupils 
attending Potter-Thomas School in 1970, the year before the Bilingual Program 
reached the second grade. 
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V. 

As can be s^^^, the raw-score means of both Model A and Model B 
pupils are great^^r than'*lthat of the preprogram base-line group on all four 
tests, with Model A pupils consistently superior to those of Model B. Some 
of the gains of Model Bjwere, in fact, so small that the scores were in the 
same **grade equivalent" range as were those of the base-line group. 

Three analyses were performed on the data to answer. three distinct 
questions. The first of 'these, Base-line vs. Combined Model A and Model B 
sought to determine whether performances of pupils in the Model School programs 
were superior to those of , the b?se-line group. Overall, for three subtests — 
Paragraph Meaning, Spelling, and Word Study Skills — the results clearly were 
in favor of the program, but the outcome on Word Meaning was marginal (F - 2.30, 
df = 1/105, p ClO) . 

The second analysis. Base-line vs , Model A , responded to the question 
whether the performance of Model A pupils was superior to that of the base- 
line group. As can be seen in the table, there were clear-cut differences 
on all four subtests. (F ranging from 4.43 to 31.03, and probabilities ranging 
from less than .04 to less than .00^) with the strongest differences in Spelling 
(one-half year of grade equivalence) and Word Study Skills (six-tenths of a 
year of grade equivalence) . 

- The third analysis examined whether the performance of pupils in 
Model 8 was superior to that of the base-line group. Analysis showed that 
program pupils were significantly 'mead of the base line in three skill 
areas — Paragraph Meaning, Spelling, and Word Study Skills — and marginally 
ahead in Word Meaning. However, examination of the grade equivalents • suggests 
that at least two of these were probabJy not meaningful, despite the fact 
that they exceeded chance expectations. Specifically, the Word Meaning and 
Paragraph Meaning performances were less than one month of grade-equivalent 
score ahead of those of the base-line. 

Performance of Anglo Model A Pupils on the Iowa Tests 

Table 8,2 shows the results of comparison of the Iowa test performance 
of Anglo Third-grade Model A pupils with that of the 1971 Potter-Thomas base- 
line group. As can be seen, Model A pupils* performance was superior to that 
of pupils in the base-line group on all four subtests and in the Composite 
score. The analyses of variance show that on one stibtest — Vocabulary — Model A 
pupils were clearly ' superior (F 10.7, df = 1/145, p<.01). On two subtests 
Language Skills and Arithmetic — marginal levels of significance (p <.10) were 
obtained. For only one subtest — Reading — were the results nonsignificant. 
When the scores are combined into the publishers* Composite, analysis shows 
a clear-cut difference of 2.5 months' grade equivalence in favor of the Model A 
Group (F = 6.69, df = 1/135, p <.01) . 

Table 8.3 shows comparison of four subgroups of- Model A. third-grade 
Anglo pupils: those who had been enrolled in the program for more than two 
years ^ those who had been enrolled for one to two years, and those admitted 
during the early and later parts of the year being evaluated. 
As can be seen from the table, there is no clear pattern of Iowa test scores 
which relates to length of exposure to the program; and none of the analyses 
of varicince revealed a statistically significant difference. 
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Table 8.1 Performance of Anqlo Pupils in the Second Grade of Model B 
on the Stanford Achievement Test, Readina Subtests. 



Sul-jtest 






5e 


Tiine 








Model A 










Mo^^el 


B 








(N 




42) 








(N 


- 39) 










(N 


27) 






X 


Gr . 




. S 




X 




Gr . 




s 


.D. 


! X 




Gr. Kq 


. S 


■ D. 


Word Meaning 


1' ■' ■ 


1 


7 


3 


8 


8. 


0 


1 


.8' 


4 


.1 ■ 


1 


4 


1.7 


4 


.6 


^^aragraph Meaning 


j 8.6 


1 


6 


6 


3 


12. 


3 


1 


.7 


6 


7 




1 


1.6 


6 


.9 


Spelling 


!2.1 


1 


5 


2 


X 


i 5. 


4 


2 


.0 


4 


,3 


' 3. 


3 


1.7 


5 


.1 


Word Study Skills 


ll6.5 


1 


4 


6, 


3 


1 26. 


5 


2 


.0 


7 


.1 


i 18. 


2 


1.5 


10 


.5 



Multivariate Analyses of Variance of Tx/lw Scores^ 



Item 



df 



Base-iine vs. Conibined Model A and Model B 



Multivariate F 
Word Meaning 
Paragraph Meaning 
Spelling 

Word Study Skills 



4.. 38 
2.30 
3.48 
7.66 
16.94 



8/204 
1/105 
1/105 
1/105 
1/105 



.001 

.10 

.03 

.001 

.001 



Base-line vs. Model A Alone 

Multivariate F 
Word Meaning 
Paragraph Meaning 
Spelling 

Word Study Skills 
Base-line vs. Model B Alone 



8.19 
4.43 
6.23 
15.05 
31.03 



4/102 
1/105 
1/105 
i/105 
1/105 



.001 

.04 

.01 

,001 

.001 



Multivariate F 
Word Meaning 
Paragraph Meaning 
Spelling 

Word Study Skills 



8.70 
2.96 
6.86 
13.83 
33.08 



4/102 
1/105 
1/105 
1/105 
1/105 



.001 

.08 

.01 

.001 

.001 



*A11 three analyses were taken from three different sets of orthogonal 
comparisons. They were created to answer the specific questions raised in 
the text. It was not possible to include all three in one data analysis. 
The author is aware that, strictly speaking, a single analysis and post-hoc 
tests would be more conservative, but it was felt that that approach would 
have been conservative for the purpose at ^and. The reader will note that 
the degrees of freedom are the same for the error variance regardless of the 
number of cases actually compared in an analysis. This is because in orthogonal 
comparisons, the error variance is pooled from all cells of the experiment. 
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Table 8,2 Comparir7on of Iowa Test Scores of Model A Third-Grade Anglos 
VJi th Scores of Anglos in Base-line Group. 



Subtest 


Model A 


Base Line 
( Potter-Thomas ) 


df 


F 


P 




N 


Mean (GE> 


N 


Mean (GE) 








Vocabulary 


■ 56 


2.70 


91 


2. 34 


1/145 


10 


.7 


<.oi • 


Reading 


54 


2.. 52 


92 


2.35 


1/144 


2 


.11 


NS ■ 


Language Skills 


■ 56 


2.53 


90 


2.33 


1/144 


3 


.43 


<;^. 10 • 


Arithmetic 


54 


2. 77 


88 


2.54 


l/:'.40 


3 


.36 


c.io 


Composite 


52 


2.59 


85 


2.34 


1/135 


6, 


69 


(.01 



■^Grade-equivalent scores were used in this analysis.. All pupils were 
tested with Level A. Univariate F_ tests were computed. 



Table 8.3 Comparisons of Bilingua 1 program (Model A ) / Third-Grade Anglo 
Pupils of Varying- Lengths . of Exposure to the Program, 



Length of Experience in Bilingiial Program 



Subtest 


More 
Two 

N 


Than 
Years 
X 


Between One 

and 
Two Years. 
■ N X 


Between One- 
half- and 
One Year 

■ N '. ■■ .X ■■ 


Xess Than 
One-half 
Year ■ 

N ^ X - 


df 




P 


Vocabulary 


29 


2.62 


• 15 


2.68 


9 ' 


2.70 


6. 


2.55 


3/55 


0.4 


NS 


Reading 


29 


2.52 


■ ■' 15 


2.45-^ " 


9 


2,60 


7 


2.38 


3/56 


0.2 


NS 


Language 
Skills 


29 


2.51 




• 2.42 


11 


2.70 


7 


2.65 


3/51 


0.4 


NS 


Arithmetic 


28 


2.80 


.15 


2.58 


■ 10 


2 .89 


7 


2:83 


3/53 


0.4 


NS 


Composite 


28 


2.58 


■ 15 


2.54 


7 


2.74 


6 ' 


2.42 


3/52 


0.3 


NS 



lunivariate F tests were computed, using grade equivalents'. 
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Second- and Thira-Grade Latino Perf orinance on the Test de Dastrezas Basicas 
en Lectura 

Table 8.4 shows the performance of Latino second- and third-grade 
pupils in the Model School programs on the four subtests of the Test de 
Destrezas Bisicas en Lectura, In both the second and third grades and on 
all subtests, pupils were superior to the base-line group of Latino pupils 
in the Philadelphia schools in 1968, with the mean overall score showing a 
superiority of 13 percentile points in the second grade and 17 percentile 
points in the third grade. 

The analysis of variance showed that the differences on all subtests 
were statistically significant. 

Table -8.5 shows that there was not a significant difference in the 
Composite scores between pupils in Model A second grade and Model B second 
grade. It also shows .that analysis of variance of scores grouped by length 
of exposure to bilingual education. did not yield statistically ' significant 
differences. It should be noted, however, that there is a trend for each 
group, with lesser amounts of exposure, to the program to have a lower composite 
score than the groups with greater exposure/ ah outcome one could expect in 
only four times per hundred bychance. This suggests that analysis of a 
different type might detect differences. - 

Latino Third-Grade Pupils Tested in English " " 

Latino pupils in the third grade were tested in English on the four ■ 
subtests of the Iowa test, in order to describe their perfonp.ance in that 
language after their entry into the Bilingual Program. Results, shown in 
Table 8.6, indicate that Latino pupils in the third grade- are working. at a 
level typical of English-speaking pupils nationally at the .end of the first 
grade. Table 8.7 shows the influence of length of time .in the Bilingual 
Program on each of the subtests. As can be seen, no systematic, statistically 
significant differences were found. 

Discussion : 

The findings of this study, show that' performance of students in the 
Bilingual Program was generally better than that of base-line groups* At no 
point was it poorer. . ' 

For Latino pupils the gains in the- measured Spanish reading skills 
have been substantial (13 percentile points in the second grade, 17 percentile" 
points in the third). This 'is not- unexpected because the base-line data were 
for pupils not, receiving instruction in Spanish; while Latino program participants 
.were receiving such instruction. Nevertheless, the fact that gains were 'strong 
and overall pupil performance (Composite score) was above the 40th percentile 
of pupils in an all-Spanish environment suggests that the program can succeed 
in developing "performance of pupils in • their, mother tongue under these conditions. 
It should be noted that in the third-grade results there is an interesting 
pattern in ^both the base-line and the program outcomes. When compared with 
norms of , Puerto Rican pupils y performance on recognition o*f Words and Letters 
(which requires little comprehension) is much better than .performance on: the 
Word Meaning and Comprehension subtests. This pattern' may provide c-lues to - 



Table S.4 Comparison of Model School Second- and Third-Grade Pupils 

with the Base Line of Latino Pupils in Philadelphia Schools 

in 1963 



'Grade and Subtest 






Base Line 




Models A & B 










(N 2t\(i) 




■■ (N 110) ■ 




Second Gr'ado 






Percent: 1 


SD 


X Percentile 


SD 


Recognition of 














Words and Letters 


43, 


.04 


35 . 


15.27 


■ 48.70 45 


16.20 


Word Meaning 


8 , 


.56 


30 


4.85 


11.6]. , 40 


6 .59 


Comprehens ion 


6, 


.06 ■ 




4.95 


7.28 40 


1.15 


. Composite 


57. 


,49 


32 


20 . 49 


67.86 45 


26.41 








Base Line 




Models A &. B 










(N = 332) 




(N = 94) 




Third Grade 


X 


Percentile 


SD 


X Percentile 


SD . 


Recognition of 














VJords and Letters 


49. 


,76 


32 


13.25 


58.39 61 


8.48 


Word Meaning 


9. 


,39 


19 


8.08 ■ 


14.25 36 


6.51 


Comprehension 


7. 


,19 


21 


7. 13 


' 10.14 . 30 


. 6.95 


Composite 


69 . 


,93- 


27 


57.94 


82.95 44 


19.37 






Multivariate 


Analysis of 


Variance 




Grade Level: 




F 




df -. 


P< 




Multivariate 




13, 


.81 


, 4/795 


• ■ .001 ■ 




Recognition of Words 














and Letters 




49 


.87 


1/798 


. 00 1 




Word Meaning • 




4 


.41 " 


1/798 


.04. 




Comprehension 




9 


.39 


1/798 


.002 




■ Composite 




18 


.00 


, 1/798 


.001 




Program 














, Multivariate 




20, 


.88 ' 


4/795 


■ 001 




Recognition of Words 














and Letters 




39, 


.12 


1/795 


.001 




V?ord Meaning 




51 


.06 


1/798 


.001 




Comprehension 




14 


.93 


1/798 


-001 




Composite 




12 


.29 


1/798 


:ooi 




Interaction of Grade Level 












and Program 














iMultivariate 




-1 


.47 


4/795 


.008 




Recognition of Words 














and Letters 




" 1, 


.72 


1/798 


■ ' . NS 




Word Meaning 




2, 


.66 


1/798 


. NS 




Comprehension 




2 


.66. 


1/798 


■ NS ^ 




Composite 




0 


•16 


. ■ 1/798 


. ■■ NS . . 





;■. . -Percentiles are for' second- and third-grade, rural pupils in Puerto 
in' the -Spring semester*. ■ 
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Table 8.5 Test De bestrezas Analyses Designed to Assess Subordinate 
Questions . - 



Comparison of Model A and Model B 
Second-Grade Latinos — Composite Scores 



Model 
A 
B 



N 

73 

39 



Groups 
Error 



MS 

11.6 
743'. 2 



6713 
66.9 
df 

1 

110 



Percentile 
43 
43 

F P 
■ .02 NS 



Comparison of Model A Third-Grade Latino Pupils 
with Varying Amounts of Experience in the 
Bilingual Program — Composite Scores 



Length of Experience 
in Program, 

More than two years 

One to " two years 

One-half to 
one year 

Less than one-half 
year 



N 

22- 
47 

.38 



■ X 

70.7 
69.8 

64.5 

45.6 



Source, MS df 

Length of time 1065 ■ .3 
/ iri Program 

Error 727 108 



F P 
1.46 NS 
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Table B.6 Third-Grade Latino Pupil Performance in English Reading and 
Spelling-Skill Subtests of the Stanford Achievement Test. 



Subtest 

' i 
J 


N 


X 


G.E. 


1 
i 

SD 


Word Meaning i 


70 


9.07 


1.8 


I 4.80 


Paragraph Meaning 


73 


13,47 


.1.8 


; 6.29 


Spelling 


5.3 


4.26 


1.9 


i 5.50 


Word Study Skills ^ 


73 


1 20.01 


1.6 


i 7.56 

! 



Table 8.7 Mean Scores on the Stanford Achievement Test of Latino Pupils 
with Differing Amounts of Experience in the Program. 



Length of Experience in Bilingual Program 









Between One 


Between One-^ 


Less 


Than 


Sxibtest 


Mpxe 


Than 




and 


half and 


One- 


half 




Two 


Years 


Two 


Years 


One 


Year 


Year 




N 


ST 


N 


"x 


N 


yC 


N 


5r 


Word Meaning 


27 


9.4 


19 


9.2 


14 


.8.6 


9 


8.7 


Paragraph Meaning 


27 


13.9 


20 


12.6 


16 


14.2 


10 


13.7 


Spelling 


24 


4.6 


11 


5.8 


10 


2.9- 


8 


2.3 


Word Study Skills 


26 


20.9 


21 


18.6 


14 


19.9 


10 


21.3 



Analysis of Variance 



Subtest 


F 


df 


P 


Word Meaning 


0.11 


3/65- 


NS 


Paragraph Meaning 


0.24 


.3/69 


NS 


Spelling 


0.86 


3/49 


NS 


Word Study Skills 


0.43 


3/67 


NS 



^Univariate F tests for each measure were computed. 
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the areas where special emphasis is needed if Latino pupils are to develop 
good mastery of Spanish while living on the Mainland-. 

Results of administi^ation of the Stanford Achievement Test for Model A 
Anglos once again showed gains were made when compared, with the base line of 
pupi.ls who previously studied at Potter-Thomas School, replicating the results 
of the previous year. The results for Model B, showing small but statistically 
significant gains (often less than one month of grade equivalence) need atten- 
tion. It was originally thought that the Model B Anglo participants would be 
like those of Model A as they come from similar schools and a similar ccmmunity, 
making the Potter-Thomas base line an adequate one. Examination of class lists 
shows t}vi>t this was not a good assumption, because the Model B Anglo second 
grade includes many (about one fourth) Spa-nish-surnamed . children. Checking 
with program supervisors showed that there was a tendency to place Latino \ 
pupils with "good" mastery of English in the classes of Model B Anglos because 
it would result in more contact with. Spanish than if they had been placed in 
regular classes. However, it also resulted in an "Anglo" group of pupils for 
whom English was not truly the first language. . 

The Iowa test data, which were used to compare Anglos at Potter- 
Thomas with the previous years *. pupils , also deserve special comment. The 
lack of clearly significant differences in areas, other than vocabulary is 
probably, due to the fact that the base- line group had also been involved in 
a special program, the District Five Reading Program, «onducted by Behavioral 
Research Laboratories. The fact that there were- strong trends (Language 
Skills and Arithmetic) and one clear-cut significant difference (Vocabulary) 
in_ favor of the Bilingual Program should be regarded as a gratifying outcome. 

Examination of the performance of third-grade Latino pupils on an 
\ English reading test designed for the elementary, grades can be viewed only 

in descriptive terms, because of -the lack -of a base line. The study showed that 
/ after about eight-tenths of a year of instruction in English, pupils .were 
reading like English-speaking pupils in a national sample with about the same 
length of experience in reading, i*e., pupils* at the end of the eighth month of 
the first grade. If this rate' of acquisition can be maintained, the .Latino 
pupils should ultimately be able to read well in the both languages. 

One outcome worthy of further study is the relationship between., 
length of exposure to the Bilingual Program and performance* It is not surpising 
that little difference was found for Anglo pupils. The results of the 'study 
of -Latino pupils reading in Spanish suggest that there may be a clear-cut 
relationship which, was undetected by the analysis of variance used but which 
might be detected by a correlational study.. It is expected that such a study 
'would show that the more exposure there was to bilingual education the stronger 
the Spanish performance would be. 

Equally interesting is the lack of any pattern in the English performance 
Latinos as a function of program exposure. Most new Latino admissions 
Bilingual Program came from other Philadelphia schools, not directly 
Spanish-speaking- educational environment. The fact that the performance 
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LgUsh envi^ilnt ^ ' ^^""^ °' ^^^'^^"^ experience in an all- 

nof di«erf-om tL\'o/i °' ^" Bilingual Program) did 

Jears of Enf??^h . l°ng-tenn participants (with one or one and one-half 
or a vefr and . '"^^""'^ accomplish in a year 

Of time to L^o^l . "^'"^ "^"i^^ ^ "-"-h longer period 

to te^t direcM . ''"^'"^ subsequent evaluations it would L desirable 

o1 ~^1iLrp\\Ll™ .elationships between length 
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STODY 9. CRITERION-REFERENT REA.OING. TESTS IN THE MODE'. A AND MODEL 3 PKOGRAr-53 



As we noted in Study 8, a major change in the evaluation of the 
Model School program v;as initiated during the third operational year — a 
greater number of standardized tests was used to assess pupil performance. 

In order to conduct rhis standardized testing, the limited evalua- 
tion resources demanded a cutback' in the evaluation in some other area. 
This cutback occurred in the assessment of reading using criterion-referent 
approaches. Therefore, in Spring 1972 assessment of reading "uhese approaches 
were confined to those groups and grade levels where the project, staff Jfelt 
that standardized testing was inappropriate.. As a result/ "criterion-re cerent 
approaches were used in the ail-day kindergarten and first-grade for measure- 
.ment of first-language reading- skills as well as with second-grade Latinos 
and third-grade Anglos for measurement of second- language skills. 

r . ■ 

Two tests of the criterion-referent battery were used — Word Calling 
and Pictures. The Word Calling test measures' sight vocabulary and was used 
with all 'groups; the Picture test' measures meaning of words and was used 
with- first-grade pupils learning to read their first- language. . 

The evaluation of the 1970-1971 school year showed that pupil 
performance on the picture subtest was highly erratic. It was believed that 
this was due to departure from prescribed methods of test administration. 
To remedy this situation, during the current year all testing was carried 
out by supervisory and evaluation staff members in Model B, and by teachers 
with direct supervision by the program supervisors in Model A. 

Procedures ■ 

Program . • 

In 'the Model A program, pupi].s in the all-day kindergarten began 
to learn to read about midyear. Anglo pupils were introduced to the skills 
with the Chandler prepriraer program while Latino pupils began to learn to 
read using the Laidlaw series in Spanish. In first through third grades, 
pupils began to learn, to read using the Laidlaw series in Spanish. In 
first through third grades.^ pupils used the Bank Street Readers in English 
and the La idlaw' Readers in Spanish.. First graders began reading "in their 
mother tongues, second-grade Latino pupils began reading in .their second 
language, and by third grade Anglo and Latino pupils were reading in both 
their first and second languages. 

In Model B the reading instruction of Latino pupils was under 
direct project control but the instruction of Anglos was not. Therefore,' 
evaluation of the reading of Model B.Anglos was riot included in the criterion- 
referent testing.. In ixiode'l B, ^second-language reading was not formally 
introduced to either the Anglo or the Latino groups. This component iased 
the Miami curric.ular materials. ' The Language Arts strands of these materials 
included activities which provided readiness skills for the Latinos learn- 
ing to read in their. mother tongue* "I'he Lippincott reading materials were 
used" in Model A and B as supplementary texts during the year for instructing • 
pupils to read English* 
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objectives 



All-day, kindergarten^ mother tongue > Kg objecti-ves were specified 
for this level. Any achievement obtained is beyond that mandated by the 
proposal. 

First grade, mother tongue . Objective 3,7 stated that 80% of -zhe 
children would be able to recognize 90% of the sight vocabulary of the 
preprimer and primer (first primer in Bank Street)' of the reading series in 
use — Bark Street for Anglos, Laidlaw for Latinos. 

Second grade ^ mother tongue . Objective 5*1 stated the following 
distribution would be obtained by the end of the second grade: Level 2f 
Part 1 of the readers would be completed by the 50% of the pupils j Level 1 
would be mastered by 30%-/ the remaining 20% of the pupils would show mastery 
of the preprimer level. 

Secondgrade, second language. Based on the. success in intro- " 
ducing second- language skills to a few pupils' in the second operational year 
all Latino pupils in Model A during 1971-1972 were introduced to reading in 
their second language. (Although. Anglo pupils who. began to read did equally 
well, it was felt that the majority of Anglo pupils did not have sufficient 
mastery of Spanish to warrant introducing them to this broad-based reading 
program) . . 

Third grad^ first language . Objective 6.1 specified the distri- 
bution of pupil reading skills expected. Pupils were evaluated informally 
in the "Log Study," (see Study 9 and with, standardized test (see Study 6). 

Third grade, second language ^ Objective 6.2 specified the 
distribution of pupil scores expected in the second- language for both 
Anglos and Latinos.' Pupil performance was checked in the Log Study (see 
Study 5). In addition, Anglos were tested in the second- language in this 
study. The anticipated performance was 60% of the , Anglos able to read 
Level 1 at 90% criterion and 40%Tkt the primer level to that criterion. 

Latino pupils seemed capable of more complex performances. .They^^ 
were tested in their second language using the standardized English- language 
test normally used with second-grade Anglo" pupils • (see Study 6) # 

In all cases word-calling criterion-referent tests v?ere specified 
for assessing the criterion. The picture tests were still regarded* as 
experimental. . ' / 

Evaluation ^ 

• Instrujnents . The instruments used in this study were developed 
in 1970-1971 for the evaluation of: .this program, and were described in con- 
siderable detail in the evaluation report for that year (Offenberg, 1972). 

Chandler Readers * Anglo pupil "performance on this preprimer 
material used^in the all-day kindergarten was measured by a word-calling 
test. It consists of ten words:*.se'iected fromlthe' in 
the first five booklets^.of this ser"ies> . Swings , slides . Trucks, and Cars 
to Ride , Bikes , and Supermarket^ . ^'Xrix^'tha testing pupils v;ere asked to read 
the v/ords aloud. . • 



Bank Street Readers This masuery of English reading skills was 
measured by tv/o of the four criterion-referent tests for these inaterials — 
word Calling and pictures* The' first of these sampled vocabulary words 
appearing in the Bank Street readers- Children being tested were required 
to read the words aloud. The testwas designed for individual aajninistration. 
The second test, Pictures, consisted of pictures which portrayed the meaning 
of concepts taught in the reader. The pupil was to match one of three words 
with each picture. The picture test was designed for group administration. 
Both tests of the Bank Street battery were designed to assess reading of 
texts from the prepriiners through Level 3.1. The specific readers were 
In the City People Read , Around the' City ^ Uptown Dcf\^ ntox-jn . My City ^ Green 
Lights Go , and City Sidev;alks . Detailed descriptions of the tests appeared 
on Page 91 of the second-year evaluation report. The v/ord-calling and 
picture tests of the Bank. Street Series were administered to Anglo first- 
grade pupils. Latino second graders were examined with the word-calling 
test for these readers. 

Laidlaw Readers . Two subtests based on the Laidlaw readers in 
Spanish were also used,, one a word-calling and one a picture test. In 
•format these tests were similar to those prepared for Bank Street readers. 
The tests themselves were described in detail in the 1970-1971 evaluation 
report. The tests measured skills in the Laidlaw readers through the 2.1 
level. The books sampled were Camino de la. Escuela ^ Aprendemos a- Leer , 
NUestros Axnigos , and Del Campo al Pueblo . 

Latino all-day kindergarten pupils were tested on word-calling 
, of the preprimer materials. Latino first graders and Anglo third graders 
were tested on the word calling. Latino first graders were tested on the 
picture subtest as v;ell. . 

Method . Two different procedures were used. The word-calling . 
tests were administered by project-evaluation and supervisory staff members. 
The picture tests were administered by teachers, with supervisors monitoring . 
the process insofar as that was possible. Program-evaluation staff monitored 
those classrooms where the teacher appeared to have departed from the pro-' 
cedure ■ for testing the previous year. 

Subjects . in each of the Word Calling tests^' a sariple of one- 
third of the. pupils in the grade levels tested was drawn at random by 
members of the evaluation staff. The one exception to this was the all- 
day kindergarten, where all pupils on roll and present during the testing 
period were examined. 

The picture tests were administered to all pupils in each grade 
level present on the day when the testing was conducted. 

Analyses . Means, standard deviations and other descriptive 
statistics were computed in order to compare pupil performances with criteria 
indicated in the objectives stated in the proposals. Tn addition, the 
picture subtest .results v;ere subjected to item analyses in which the , point- 
biserial correlations of • items with total score, kuder-Hichardson reliabil- • 
ity. coefficients, and other test-pertinent data were obtained. 
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W ord Calling Tests 

. The all-day -kindergarten pupils^ as noted earlior, bo jan to read 
preprimer texts in their mother tongue. Although no readincj objectives 
v7er'=^ ever stated for this:, group / pupils v;ere able to read this level *^of 
niaterial by year end. As can be seen in Tables 9.1 and 9.2, typical 
Aiiglo and Latino pupils were able to call about 70% of che vocabulary 
of the texts to which they were exposed. 



Table 9.1. Performance of Anglo All-Day Kiiiaor^jarten 
Pupils on the Chandler Reading ^est (iVlodel A) 



Mean Score 
Standard Deviation 
Text Maximum 
Criterion 

No. of Pupils Observed 
Percentage of Enrolled 
Pupi Is Obs er ved 



7.0 
2.7 

10 

None set 
17 
100% 



Table 9.2. Results of Testing All-Day Kindergarten 
Latino Pupils on vhe Preprimer Iterrs of Laidlaw 
word-Calling Test 



Mean Score 
Standard Deviation 
Text Maximum 
Criterion 

No* of Pupils Observed 
Percentage of Enrolled 
Pupils Observed 



The performance of Anglo pupils i i the first grade on the Bank 
Street Word-Calling tests is shown in Table 9.3. As can be seen only 32% 
of the pupils met the criterion. According to the objective, 80% of the 
pupils should have met it. The performance of tha Latino first graders in 
the Laidlaw text was also less than that st-ited in the objective. As shown 
in Table 9.4, only 28% of those in Model A » nd 15% of those in Model B were 
able to complete the word-calling test at tue level of the criterion. 



7.0 
2.5 

• 10 

None set 
22 
100% 
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Table 9,3. Tfisting of Anglo Model A Firsr-Grade Pupils 
on Word-Calling Test for lank Si:reet 



Mean Score 


I'D. 31 


.S'tandard Deviation 


11.43 


^o, of Items on Test 


(\9 


Ko. of Items Applicable 


1:0 


to Objective 




Criterion (90% of previous 


18 


Dine) 




NOo of Pupils Observed 


jl 


Percentage ox Observed Pupils 




Meeting Criterion 





Table 9,4. P-jrformanca on Model A and Model B Latino 
First-Grade Pupils on Word-Calling Test 
for Laidlaw (Spanish) Readers 





i 

Item 1 


Mode] A . 


Model B 


Mean Score ' 


13.42 


10.45 


Standard Deviation 


12.33 


10,63 


No. of Items on Test 


40 


40 


No. of Items Applicable 


20 . 


20 . 


to Objective ' 






Criterion (90% of previous 


18 


13 


line) 






No. of Pupils Observed 


64, 


47 


Percentage of Enrolled 


38 


39 


Pupils Observed 






Percentage of Observed 


28 


15 


Pupils Meeting 






Criterion 







Second-grade Latino pupils v.ere tested in word calling in English. 
No objective with a specific criterion appears^ but the results shown in 
Table 9.5 indicate that the second-grade Latino pupil5. were able to read 
24.83 words, which would place them about midway throu'^h the Level 1 reader 
in the Bank Street series. 

. Anglo pupils in the thirrd grade were similarly tested in their 
second language. Accordin^j to the proposal 60% of the pupils ought to 
hav^^ been able to call words in the vocabulary appearing in the Level 1 
text. Results obtained in this testing (shown ii\ Table 9.6) were far below 
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Tabic y.^* Ccscond-Language ?erfox:\ance of Model A 
Second-Grade Latinos Reading Bank Street 



Mean Score 24.83 

Standard Deviation 18*76 

No. of Items on Test 69 

No^ of Items Applicable Not Specified 

to Objective 

No, of Pupils Observed . 

percentage of Enrolled 27% 

Pupils Observed 



Table 9#6# Second-Language performance of Model A 
T*hird-Grade Anglo Pupils on 
Laidlaw Word-Calling "rest 



■ 

■ 

Mean Score 


11.86 


Standard Deviation . 


10*51 


No, of Items on "'est 


40 


No. of Items Applicable to Objective: 




Primer 


20 


Level 1 


30 


No. of Items Applicable to Objective 




at "90% Criterion: 




Primer 


IB 


Level 1 


27 


Percentage of Pupils Meeting 




Criterion: 




Primer (100% anticips^ted) 


■ 27% 


Level 1 (60% anticipated) 


9% 


No. of Pupils Tested 


22 . 


Percentage of Enrolled Pupils Tested 


34% 



that level. With only 9% of the Anglo pupils able to perform at that 
criterion^ The proposal also stated that all Anglo pupils should have 
been able to read the preprimer and primer. The result was that 27% 
of the pupils tested could read it. 

picture Tests 

Bank Street picture-test performance of Anglo first'-grade pupils . 
The entire Bank Street picture test was completed by 98 of the 168 pupils 
on roll in the first grade (58%). It was expected that all Anglo first- 
grade pupils not absent oh the testing, dates would be examined. It is not 
clear at present why so many gupilsi- were excluded, since with normal absences 
about 80^ of the pupils shquld;.h^ye\tak^^ test. Ov^ralil the it 3m 

-analysis of this test provLdes'. th'^'^'iffiij^ression that it is fairly reliable* 
The Kuder-Richardson 20 reliability coofficient wts .67 and -lost items had 
medium-value point-biseria I correlations with total sccre. (They- ranged 



from .04 to ,45, with most in the upper half of that range*) The typica]. 
item was answered correctly by 47% of the tested pupils. 

The mean score on the test was 14.1 items correct. Six items 
of the test were culled directly from the preprimer and primer levels. 
As there were 18 items on the test and three choices per item, the average 
score. of who could complete each item in the recommended texts and guess at 
the remainder would be 10 items. The 14.1-item mean reflects achievement 
beyond that ascribable to guessing and mastery of the items of the preprimer 
and primer levels. The teacher of these materials ascribed this outcome to 
the fa .t that most alternate (incorrect) choices had different initial con- 
sonants. The teacher felt that when they could guess the appropriate word 
for the picture, the pupils could look for a word with the same initial 
consonant and thereby select the correct choice, without being able to read 
the words completely. 

Laidlaw picture test with Latino first graders . . Two hundred two 
first-grade Latino pupils completed this test — .133 from Model A and 69 from 
Model B. This represents 79% of the Model A roll at year end and 86% of the 
Model B roll at year end. However, 20 Latino pupils, all in Model A, were 
examine'd on only one half of the test, reducing the Model A pupils to 113. 
Results of the item analysis, based on responses of the remaining 182 pupils, 
showed that 10 of the 12 test items had positive, point-biserial correlations, 
eight of which were .30 or above. The two items which had negative correla- 
tions with total score correlated about -.30, but a large proportion of 
pupils answered the items correctly (over 65% in '^ach case). The average 
correlation of items was .31; the typi .^al item wa3 coirpleted correctly by 54% 
of the pupils; the Kuder-Richardson 2C reliability was .79; the mean score 
was 8.35 items; and the standard devic tion vas 2.99 itoms. Six of the 12 
items came directly from i:Jie material: stuc'ied. If the pupils could complete 
each of these items and guessed on th< remainder, the results would be just 
about those obtained. However, as was the case with the Bank Street picture . 
testr the difficulty of the items, did not appear to be clearly dependent on 
the level of the reading texts from which they came. The mean was about 'one 
more item correct in Model B than in ^ Ddel A. The mean of Model A was 7.97 
items correct; that of Model B was 8.^ 7 items correct. Examination of the 
individual scores showed that there w. s a fair number of pupils with perfect 
scores Mabout one-fourth) and a fair amber of pupils (about one-*eighth) 
who made the' same response (e.g., always cl: ose the middle one of the three 
choices) in eac group. 

Discuss ion 

The results of this study must be examined in the context of 
other reading assessments in the program. However, the outcomes suggest 
that greater attention needs to be given to reading in the Bilingual program 
if pupil achievement is to match that indicated in the program's proposal 
and objectives. 'The three groups where no criteria for reading were speci- 
fied (Anglo all-day kindergartners in the mother tongue. Latino all-day 
kindergartners in the mother tongue, and Latino second graders in the second 
language) all appear to have turned in respectable performances. 

In other parts of the program studied using a criterion-referent" 
approach, the results are a complex mixture of outcomes which, taken together. 
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represent performance below that of the objectives of zhe program. 
Relatively few Anglo and Latino first graders met the first- language read- 
ing objective set for them — the mean pupil performance v;as at about the 
level specified for the minimum pupil performance. As was the case last year, 
the variability of pupil performance (as shown by the standard deviations) 
was large when compared to the mean, ]3oint:ng to skewed distributi'bns in 
which many pupils had relatively lov; scores and a few pupils had very high 
ones. 

Hhe analyses of the picture subtest suggest that rather than 
measuring a more comniex skill than word recognition, it measured a less 
complex skill — recognition of initial consonants. while it may be useful for 
that purpose, it did not seen, to be a good tool for measuring understanding 
of the written word as originally intended* 

The increase in use of standardized tests in the program suggested 
that at the earliest skill levels reading may be best measured through the 
simple word'-calling tests and more complex skills may be best measured by 
published tests. 



ERIC 



158 



: study 10-- CRITERI0N'~R2?EREK? THSTIN'G GF SECOND- AMD THIRD-GR2Jo:iE ARITHMETIC 
SKILLS, m THE MODEL SCHOOL PKOG'i^M 



During. the 1970'-197r school year , criterion-referent tests in 
arithmetic* were developed in order that second-grade pupil performance could 
be described in terms related to the Philadelphia .Levels' of Mathematics for 
the Primary Years (School District of Philadelphia, 1368) . Results of admin- 
istration, of these tests appeared in Study 8 of last year's evaluation report 
(Of f enberg^ 1972) , . They showed that Anglo pupils were more spread-out than 
anticipated, with many pupils scoring higher than stated in the objective 
and many pupils scoring lower than expected. In contrast, all Latino pupils 
•scored above anticipated levels. 

This study is in part a replication of the one conducted last 
year'. A hew set of arithmetic problems was developed for assessing pupil- 
performance in the third grade, which permitted the evaluation approach 
.used in the second grade of 1970-1971 to be used at the third-grade level . 
this year. - 

As there has never been a -formal evaluation of the .criterion-referent 
arithmetic test, an item analysis was also added to this year's-study of pupil 
performance. 

Objectives 

Second Grade (Model A and Model B) . When examined with a brief test 
sampling materials appearing. in the texts /; pupils will be at the following 
levels of the Philadelphia arithmetic curriculum: 

.At least 30% of .pupils "will; be doing satis factoiry. work;, 
at about Level ' 6 . . 

. .At least 60%. (cumulatively 90%) ' of the pupils will be ^ ■ 
■ ^ ' doing satisfactory work at about Level' 5* ' 

■.Not more than 10% will be working at about Level 4. 

. • ' . Third Grade (Model A).' . When examined v;ith a brief test sampling 

materials appearing in the texts, pupils will be at the- following levels -of 

■the Philadelphia arithmetic curriculum: ■ . . ■ . . 

- ' ■. ■ ' . ■ 

.At , least. 30% of the pupils will' be at Levels .8 and 9 (Level • . 
9 is a review) . . • 

.At least 20% (cumulatively 50%) of the pupils will be. at 
least Level 7. . ■ ■ 

.Not, moire than 50% of the pupils will be' at about - Level 6; .. 



..To .be regarded as having mastered a level, pupils will have , to answer 
correctly 75% of all; test items' up through that level. ' 



J 



•procedures 

Program 

Mathematics was first taught in the pupdis ' mother tongue. Some 
concepts were then retaught in their second language. In both -instances-, 
the SR7i Mathematics Series in both languages were used. The procedure was 
similar for the second- and third-grade classes, except that the third- • 
grade pupils had greater contact with their second language {about 25% of 
•the time) . Oral testing in mathematics in both languages was conducted as 
part of the Log Study (S.ee Study 5). In this assessment, the pupils were . 
tested in their first language. ■ 

Evaluation , 

Instrument . ■ The arithmetic test used in assessing these objectives 
had three parts / which are itemized in the Results pf this report. The first 
two parts examine pupil performance through Level 7 of the Philadelphia 
curriculum and were used during the 1970-1971 school year with -second-yrade 
pupils. The "vhird part', containing items from Levels 8 and 9 (review) ,| was 
developed this year. Second-grade pupils were tested. on the first two parts, 
third-grade pupils on all three parts. The tests were developed by the teachers 
and program supervisors who reviewed the arithmetic text and the Philadelphia, 
curriculum guide and constructed .relevant items in the same manner as last 
year. 

Method. All tests were administered by' the classroom teachers to 
pupils in therr classes. Anglo and Latino pupils received test copies with 
instructions in their mother tongues . Teachers 'administered the three parts " 
at their convenience over a three-day period. All tests were administered 
during May, 1972. 

Subjects . All pupils in the class were to be tested. However," there 
was not time for* individual testing of pupils who were absent. The percentages, 
of enrolled pupils who were' tested, at each level were as follows: 

/ ; ;Model A .■ - . . 

Second-grade Anglos -84% 
Second-grade Latinos 73% 

Third-grade Anglos 90% ■ " / , ■ , 

Third-grade Latinos , 71% 

Model B . '. ; _ -y 

■. ■ Second-grade Latinos 82% ; 

As the teaching of arithmetic to Model A.Anglos v^as not under project- 
control, - they . were not tested as part of this, evaluation. / > ■ , 
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Analysis . The performances of various pupil groups were compared 
with the levels specified in the objectives. Item analyses consisting of 
point-biserial correlation of the pupils' success oh the item with their 
total test scores, Kuder-Richardson 20 reliccbility , and the percentage of 
pupils succeeding with each item were computed for second- and third-grade 
pupils' tests. 

. * Results 



Table 10.1 shows the results of the item analyses. ' Overall results 
were as expected. Correlation of all items but two (in the third-grade 
results) with total score are significant at the .05 level (i.e., all are above 
.15 in the second-grade data and. .17 in the third-grade data) . The Kuder- 
Richardson i^eliabilities are satisfactory-- . 86 and .93 for the two test groups.. 
Examination of the percentages of pupils; with correct answers shows a common 
pattern. throughout both test versions: addition and multiplication items which 
require carrying, subtraction requiring borrowing, and division with remainders 
proved most difficult for pupils.- This suggests that many pupils have difficulty 
working with the concept of place value and with uneven outcomes in division. 



'Tai3le - 10 . 1 I-teia Ai-ialysis - of -tiie* Critrerion-Ref erent^Ax'ixhmeTic Test- 



Second Grade 
(N = 144) 



Third Grade 
(N = 123) 



Percentage of Point- 
Test Item Pupils Answering Biserial 

Correctly Correlation 

with Total 
■Score 



Percentage of 
Pupils Answering 
Correctly 



. Point- 
Biserial 
Correlation 
with. Total 
Score 



Part I 



1 
+6 



96.5 



.416 



97.6 



237 



2 

2 +9 



89-6 



,394 



94,3 



.253 



.20 
3. 4. 6 



87.5 



, 416 ■ 



86.2 



,404 V 



• 38 
4. V .4 



"43.2 



.435. ' 



79 .-7 



522 



42" 
5.: +28 



36.1 



,597 



59.3 



569 
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, Table 10.1 (Continued)' 



. 26- 

7 

•6. +65j 



16.0 



.499 



39.8. 



.520 



14 

> . 15 
J. v+18 



20.1 



,370 



62.6 



.672 



8. 4+3= 



78. S 



.173 



86.2 



.503 



9 

9. -4 



76.4 



.413 



88.6 



.456 



45 

10. -3 



45.1 



.497 



64.2 



.632 



' ' 37 ■ 
11. -21 



43.1 



.519 



69.9 



.536 



90 

12. . -45 



8.3- 



,237 



32.5. 



.619 



65 

13. -32 



43.1 



,543 



72.4 



.538 



100 
14. -64 



12-. 5 



.560 



19, 



.464 



138 
15. -69 



6.3 



, 423 



19.5 



.514 



Part II 
16. . Time = ■ 8: 30 
17 . Time ,= 3 : 00 



48. 6 
79.2 



.575 
,381 



56.9 .638 
93.5. •:.209 
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Table 10.1 (Continued) 



1.8. 7 

x6 



lb. 7 



.443 



50.4 



.648 



•19. x6 



20.1 



,518 



53.7 



.644 



4 

2b. x8 



21.5 



,525 



53.7 



.659 



21. 8^16 



22. 7 



20. 8 



29.2 



.539 



.562 



51.2 



49.6 



.700 



.590 



23 



. 6; 36 



17.4 



.553 



52.8 



.587 



24. Color 1/2 of 
. . rectangle 



.95. 1 



181 



91.1 



.069 



25. Color 1/2 of 
triangle 



.90.3 



. .333 



92.7 



.128 



26, Color 1/3 of 
circles 



83.3 



,147 



81.3 



.306 



21\ Color 1/3 of 
rectangle 



81.9 



.405 



93.5 



.190 



28. Label parts of, 

rectangle divided 
into three 64.6 



■34.4 



36.6 



.487 



29. . Label parts of a 
circle divided 
in two . 51 . 4 



,534 



60.2 



.361 
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Table 10.1 (Continued) 



Part III:/ (Third Grade Only) 



30. Write the Roman 
Number 2 



63. '4 



.452 



31, Write the Roman 
Number 5 



60.2 



.497 



32. . Write the Roman 

Number 16 ' 



43.1 



.464 



16 

33. x3 



37.4 



— .555 



23 

34". x4" 



4.1 



.389 



35. 5 46 



7.3 



.265 



36. if 



30 



6.5 



.273 



35 

37. +27 



52.8 



.616 



23 

38. +40 . 



68.3 



.583 



267 
39. +132 



67.5. 



.511 



518 
40. -365 



28.5 



.524 



Item 


Second. Grade 


Third Grade 


Number of items ■ 


29 


■ 40 . .. ■■ 


Mean Number of Items Correct ; 


14. 


35 : 


- ,23. 


79 


Standard Deviation 




27 


■ 8. 


45 


Kuder-Richardson 20 Reliabiiity 


■ ' ■ ■ -6. 


86 


• 0. 


92 . 


Standard -Error of Measurement , 




00 . 


2. 


37 



Results in Table 10.2 show that the relationship between project 
goais and pupil performance was different for the subgroups in the Model 
School Programs. In three subgroups (Latino Model A Second Grade, Anglo 
Model A Third Grade, and Latino Model A Third Grade) pupil performance was 
more spread-out than .anticipated by project planners when they wrote the 
objective. Some students were found performing better than the objective 
specified; others (16% of the second-grade Model A Latinos, 22% of the third- 
grade Model A Anglos, and 9% of the third-grade Model A Latinos) v/ere performing 
less well than the minimum specified in the objective. Latino second-grade 
Model B pupils were at or above the minimuiTi levels expected (nearly one-third 
scored higher than anticipated in the objective) . Model A Anglo second-grade 
pupil performance was close to the levels anticipated, the main discrepancy 
being that 24% of the pupils who were at level four should have been at level 
five. 

Discussion 

Taken as a whole, the results of this year show that the exhibited 
pupil performance was close to overall expectation, but there was a tendency 
for some groups of pupils to be more heterogeneous than was anticipated- This 
outcome was similar to that found during the second operational year. However, 
to focus upon this outcome would be to ignore the findings of the item analyses 
which might provide greatest usefulness in improving the teaching in the 
programs. These item analyses showed that pupil difficulties are localized, 
especially in the area of understanding operations involving place value, 
borrowing, and carrying- Review of the item-by-item tally of pupil success in 
the previous evaluation (Offenberg, 1970) shows that similar difficulty may 
have been reflected in the previous year's results. 

In other parts of the Philadelphia school system, uneven rates of 
mathematics-skill growth has lead to the development of the Continuous Progress 
Primary Arithmetic Levels, which guided planners in the statement of objectives 
for this program. in specifying the levels, it was found that operations 
involving place value were among the most j^iifficult concepts for pupils to 
master. Thus the skills which were most difficult for Model School program 
children to master were also tlie one^ which were difficult for .pupils in the 
Philadelphia schools i'n^eneral. A special . emphasis on materials and teaching 
procedures to improve mastery of these concepts is clearly indicated in the 
Bilingual Program as well as in the entire Philadelphia School District. 
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study 11. EXPLORATORY TESTING " OF "ARRIBA" JUNIOR HIGH .SCHOOL STUDENTS 



At the middle of the third operational . year it. became apparent to 
project. management that testing of ARRIBA pupils on norm-referent measures, 
especially in reading in Spanish and English, would be useful for project 
■planning and curric?alum development. If the' testing were undertaken, it would 
clarify the skill levels which "could be demanded 'by materials prepared for 
use in the project. Specifically, it would answer the question, "To what extent 
can written materials be prepared for students to read in. English and for 
students to read in Spanish?" As the greatest investment in curriculum-materials 
development of the ARRIBA program has been and continues to be foi: the junior 
high school; it was decided to test pupils at that level, 

■ An unpublished study conducted in 1965 by the Office of Research • • 
(Desing, 1969) fortuitously provided base-line data' regarding reading skills 
in Spanish. In that study, all .Philadelphia pupils who were* Spanish-speaking 
■were examined. Junior high school pupils were examined' with the Inter-American 
Test Level 4, CEs , which measures reading skills . The 1965 results of this 
•test .served as a base line . against which' current- pupil performance in the 
program was compared; This base lin= enabled the evaluation to provide both 
a description of pupil performance in the program, and a clear way of determining 
what impact the Bilingual Program has had' on pupil performance. . \ 

The ARRIBA pupils also were examined with a variety of levels of the 
Iowa Tests of Basic Skills- Although this test is used systemwide, Spanish- 
speaking pupils who did not (in their teachers' opinions) know enough English 
to be tested were excluded in past administrations. Thus pupils most like 
those served by the ARRIBA program were excluded from previous administrations 
of this, ins t^rument , As a result, there is no reasonable base line against / 
which the- performance of ARRIBA- junior high school pupils could be compared 
on this instrument. 

Procedur.es 

Program , 

The" performance of pupils on the standardized tests cannot be attributed- 
directly to any elements in the program because it is the result of the < 
cumulative impact of (£) the instruction pupils received in the Philadelphia 
■ schools before they entered in the Bilingual Program, (b_) the instruction pupils 
were" receiving in their mother culture before coming to Philadelphia, (c) any, ■ 
previous experience students may haye had in the Bilingual Program, . and (id) the 
experiences that pupils had .during the year in which the testing ^.was conducted. 
Of . these, information exists only for the last two elements. If a-pupilhad 
received instruction in. the ARRIBA elementary grades (seventh graders one year 
ago,, eighth graders two years ago) , reading in Spanish was a distinct subject- 
in the program (with Laidlaw. readers in common use) . The "English as. a Second 
Language" program in the eOiementary grades introduces pupils to reading after 
adequate oral-aural, control is developed. The reading., elements are based 



primarily on the English as a Second Language texts and materials chosen by 
the teacher, with some supplementary use of regular English-language readers 
when pupils become ready for them. In the junior high school, reading and 
other language-arts skills are. integrated in the '^Spanish as a First Language" 
and "English as a Second Language" curricula. 

In addition to language and reading skills, the Iowa Tests of 
Basic Skills (ITBS) include a mathematics subtest and a study-skills subtest. 
The instruction in mathematics and social studies, where most of the study 
skills would b2 taught, has been primarily in Spanish; the test, of course, 
is in English, and is verbal in its orientation. Thus, for the target group 
of this study, the ITBS mathematics subtest and study-skills subtest performance 
must be regarded as amalgams of (a^) pupils understanding of concepts and (b) 
use of the English language. 

Evaluation 

Instruments . The assessment of Spanish reading skills was made 
through administration of the Inter-American Test of Reading, available in 
Spanish and English versions. The Spanish version for the junior high school 
(Level 4, Form CEs) was admini:3 tered. It provides four scores: Vocabulary , 
Level of Comprehension , Speed of Comprehension , and Composite . Norms for the 
tests are based on an Islandwide sampling of pupils in Puerto Rican schools- 

The Iowa Tests of Basic Skills (Form 3) were used to test pupil per- 
formance in the English language. As out-of -level testing is in common use 
in the Philadelphia schools and as ARRIBA pupils were selected because they ■ 
were Spanish-dominant, a different testing program was conducted at each 
school site. It had" been planned that pupils at both schools would be tested 
with the form fc one grade level below their actual grade. However, one 
school's testing supervisor felt that it would be desirable to administer . 
lower level tests because he was sure those originally specified by the evaluator 
were too dificult. Therefore^ at Penn Treaty Junior High School, the fourth- 
grade test was used with the seventh-grade ptipils., and the .sixth^grade 
test was used with eighth-grade pupils; at Stoddart-Fleisher Junior High School, 
the sixth^grade test was used with seventh-grade pupils,, and the seventh-grade 
test was used with eighth-grade pupils. As this testing was conducted as part 
of the regular School District testing program, ninth-grade pupils, who do not 
participate in the School District testing, were not examined. 

The Iowa tests yield six scores — Vocabulary, Reading, Language Usage 
{spelling, capitalization, punctuation, and usage) , Work-Study Skills (map 
reading, graph and table reading, use of reference materials) , arithmetic, and 
Composite. 

Method . The Iowa Tests of .Basic Skills were administered to pupils 
in their classrooms by classroom teachers, in aiccordance with the procedures 
used elsewhere in the School District, during the last week of April and the 
first week in May. ' The Inter-American Test of Reading was administered by 
a member ot the research staff over a two-day period during the middle of May, 
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Subjects . All 107 seventh- and eichth-crade ARRIBA pupils on 
roll in March were potential subjects for the Iowa tests: 87 pupils (81%) 
were tested. All 158 pupils in the seventh^ eighth, and ninth grades were 
potential subjects for the , Inter-American tests; 124 (78%) conpleted the 
tests. The actual number of pupils tested was dependent upon those attend- 
ing school on the days when tiie tests were administered. 

Analysis . With the Iowa Test/ the lack of clear-cut base lines 
and the variety of forms used prevented any analysis oeyond the presentation 
of descriptive statistics (means and standard deviations) which were compu- 
ted for each test level and treatment group. 

With the Inter-American test, the uniformity of test form and* the 
existence of a base line perr.iitted comparison" of program-pupil performance, 
with the (preprogram) 1968 base line (seventh- and eighth-grade students) . 
An analysis of variance on the composite scores was computed to test the 
statistical significance of the observed differences. Descriptive statistics 
also were computed for theze grade levels and for the ninth grade as well. 



Results 

Iowa Tests 

The performance of seventh graders 'on the Iowa tests is shovm in 
Table 11.1. It presents an ambiguous picture. Results ware uniform within 
the four pupil groups in that the grade equivalent earned was similar on all 
subtest Sr without large de sc rep an cies among Vocabulary , Reading, Language, and 
Work Study Skills. Seventh graders (at Penn Treaty) whose performance was 
assessed with Level B (the fourth-grade test) had a Composite grade equivalent, 
pf 2.8, with all -^f the subce sts in the narrow range from 2.4 (Vocabulary) 
to 3.0 (Work-Study Skills).. For Vocabulary and Reading, where- the guessing 
scores are easy to compute (because every item has four responses), a pupil 
who had guessed at every item could have grade-equivalent scores of 2.6 and 
3.0, suggesting that actual pupil performance- -2 . 4 and 2.7 — was similar to 
that obtainable from guessing at every test question. 

Seventh-grade pupils (at Stoddart-Fleisher) were examined with Level 
D (the sixth-grade test) . Pupils * grade-equivalent C6mposite scores averaged 
4.9, with subtest scores ranging from 4.3 (Language) to 5 . 1 ' (Vocabulai.y} . The 
guessing level is 4.1 for Vocabulary and 4.0 for Reading, suggesting that the 
actual pupil perfarmance — 5.1 and 4.8 — was only *:-lightly above that of chance; 
•pupils answered correctly only two to four questions more than pure chance 
would. have allowed. 

Eighth-grade pupils who were tested with Level D (at Penn Treaty) 
did not perform quit^- so well as the seventh graders whd took the same test. 
The mean Composite score was 'i.l, and subtest scores ranged from 4.2 (Arithmetic) 
■'to 3.5 (LancTuage) . Compari-*Honb of the Vocabulary and }^*='ading scores with chance 
levels described above suggest that pupil pex^formance was at about chance levels. 



Table 11.1. ARRIBA Junior High School Pupil Perfornarce on the Iowa Tests 
of Basic Skills. ' ' 



• Work-- 

Group & Statistic Vocabulair/ Reading Language Study Arithmetic Composite 



Grade 7 Level B 

Mean G.K. 2,19 

No. of Students 25 

Stand. Dev. .66. 

. Test Floor (zero) .1.3 



2.66 

25 

.85 
1.3 



2.70 

23 

.71 
1.3 



3,04 

25 

.90 
1..4 



2,92 

27 
1.07 
1.4 



2.78 
23 
.68 



Grade 7 Level D 

Mean G.E. 5.09 
No. of Students 16 

Stand. Dev. 1.12 

Test Floor 2.1 



4.84 
16 
.89 

2.2 



4.27 

14 

.68 
2.0 



4. 56 

17 

.69 
2.3 



4.91 

16 
1.27 
2.4 



4.88 
14 
.74 



Grade 8 .Level D 

Mean G.E< 3 -93 

No. of Students 29 
Stand. Dev. .86. 

Test Floor 2.1 



3.96 

29 

.76 
2.2 



3.52 

28 

.40 
2.0 



4.19 
26 . 
.89 

2.3 



4.58 

26 
1.07 
2.4 



4.07 
• 24 
.59 



Grade 3 Level E 

Mean G.E. . 4.35 
No. of Students 16 
Stand. Dev. .95 

Test Floor 2.6 



3.r3 

16 
1.46 
2.6 



4.28 

15 

.94 
2,4 



4.64 

15 

.78 
2.7 



5.17 

16 

-.8 
2.9 



4.48 

15 
7 



Eighth-grade pupils who were tested using Level E (the seventh-grade 
test) had a~ mean Composite Grade-Equivalent score of 4.5/ with individual ' tests 
ranging from 5.2 (Arithmetic) to 3.8 (Reading). Chance score for pupils who 
answered every question on the Vocabulary subtest is 5.0, which is above the 
score earned (4.4). . Chance score on the Reading subtest is 4.8, which is also 
above the 3.8 score earned. ■. 



Inter-American Reading Test (Spanish) 

Results of administration of the Inter-Americari test and comparisons 
with base line are shown in ^able 11.2. Pupil performance in the ARRIBA 
seventh and eighth grades was better than could be expected from guessing and 
was statistically significantly greater than that of the base-line group. The 
base-line group was at the 8th and 9th percentiles while pupils in the ARRIBA 
program were at the 25th and 29th percentiles. The analysis of variance 
shows that the difference in performance between the program and base-line 
groups would occur by chance less than one time per thousand. The analysis 



Table ii-.2. 



Kesuits of Testing Pupils on che Inter-Aner ican Test of 
Reading^ Level 4, Form CEs . 



Percentile Rank SD 

in Puerto Rican 

Norms 



Base Linf: 

7th 3r/de . 98 20.1 9 12.3 

8th Griifle 82 -24.1 8 .14.3 

i 

Program / . 

I • ■ ' . 

7th Gride 41 26.7 - 25 12.3 

/ 

8th ^^-ade 46 33.5 29 17.6 



Analysis of .Variance (Raw Scores) 



Sou/':e MS df 



Gr«iVe . 1937.0 1 9.e8 <.01 . 

' 1 ' ■ . ■ * ■ . ■ ■ 

Program 3787.8 1 18.93 (.001 

I^r^eraction 115.1 1 0.57 NS 

) 

l/ror 200,0 263' 

. / '. . '. , . 

/ ■ " ■ 

"I . 

i ' ; ' ■ 

■/rlso showed that the test was successful in discriminating- grade level. Although 

/there was no baseline with which to compare results, 37. ninth-grade students 
/also were tested. Their mean score was 28.51 (9th percentile), with a standard 
■ deviation of 11.19. —-^^ 
/ . ' - ■ > ■ - ■ 



Discussion 



The data rjar-hor^^d i.sing the Iowa test suggest that pupil perf jrmance- 
in English in thf-' A7kRI3A procrarn cannot be reliably assessed by any of the 
levels of this instrument tr.' ect in this ?tudy. The cact that -pupils' .'-.cores 
hover near che guessing leve: over the entire adminis-::ration suggests tJnt 
performnce will not be reli£.b\y measured by this ins:rume' t. Therefor^?, it 
is fei.t that formal testing of academic performance in the English language 
wou.'.d ,>robablY be served bes< by (a_) instruments designed to measure specific 
skills taught m that language (i.e., criterion-referent) or (b) instrv.ments 
design-id especially for use ^;:th non-English spec^kers first learning Er.glish. 

Results of the Inb.-r -American Test of Readir-j in Spanish are more 
encou:* iging, although, not wir.hout some ambiguities. Tho tesc administrred 
was, first, the one desig*^^d for the grade level of me participants , t.huc 
eliminating the need for use of grade equivalents as descrip:.^' rs and permitting 
pupil performance to be described in t-rrms of percentile rank. 

The =inalyses ot va.:iance of ^'^.venth- and e "hzh-grade scores showed 
that test scouTes in the program were beyond chance levels, and ^hat t7*e test 
could clearly discriminate maturation of pupils in the seventh anc' eighth grades 
as well as detect differences, in Spanish . reading skills brought about by the 
Eilingual Program at these levels. " The dlsappointi:it; aspect of the Inter-. 
American test data is in the ' performance of the nirit-Vi-grade students. In terms 
of rav; score their performance was slightly less thai that of eighth graders; 
in teinnas of percentile rank, they fell behind the otaer grade-level groups ir. 
the program. 

These results seem to suggest that the Bilingual Program was clearly 
effective in helping seventli- and eighth-grade Latino students grow in their 
mother tongue, but at the ninth grade that growth seemed to stop. 

This study was co);ducted, in part, .in order to see the type of 
.Leading levels in English and Spanish that curriculum -developers could 
anticipate* The English-language testing does not Dead to a clear answer. The 
Spanish testing does show that if it is kept fairly simple, most ARRIBA junior 
high studen:-s should be able to handle reading material designed for seventh- 
and eighth-grade students whose performance is in tlie 25th-to-30th-]iercentile 
range on norms for Puerto Rico. 



ERIC 



172 



STUDY ]2. !-;r::PT/r(:ATiori op FirviDiNas AS^iESSim riiic "ARfibA" ccwonent's 

TxMPACT ON UILKMKKTARY SCHCjOL PUPILS' ATTKNCArCCi. AND MIGH 
SCHCXDL PUPILS' DRuPOUT RATE 



Stucjics conducted during dirj firf5t two years of project opera- 
tion oxainined pupils' grades, behavior ratings, att'^ndance , lateness, and. 
dropout rates. This was doiie in two ways. Duri^.g the program's first 
year pupils* behaviors were compared to their own behaviors the previous 
year. This analysis had the potential to provide .tiis leading data because 
it left uncontrolled both changes i.. the variables" exanune'l due to 
maturation of the students and statistical regression effects which might 
havo arisen due to the selection, i.e., the inclination of school 
..personne] to recommend for eniollment in t±ie bilingual procrram Spanish 
speakinq students who had performed poorly iu r* juxd:: classes the year 
before. During the second year the analysis was mcdifie'^ in that program 
participants were compared with other students who hc^>d been enrolled in 
the same grade in regular classes in the Spring of 1969, the year prior 

the inception of the program. This method eliminated biases which 
might, be associated with differences in grade level (or maturation) but 
generated a bias of its own: as the project evaluators did not have 
much information about the base-line group, students selected for it 
were chosen on the basis of their mother tongue, as listed in SKYDAS , 
the School District's pupil-in'formation system. In many cases, these 
listings were based on pupil birthplace or pupil surname, and there- 
fore contained more students who were competent in English than did 
the ARRIBA program. As a result, the second-year base line might have 
been biased in that the control population might have been academically 
better than those pupils enrolled in the Bilingual Program's ARRIBA 
component. Grades and behavior ratings in the program were found to 
be superior to both base line groups used for comparison in the first 
and Second years. This outcome suggested that grade and behavior-rating 
results are probably real, consistent outcomes of the program. Detailed 
examination of them is, therefore, no longer necessary in every year's 
' evaluation. Similarly, it was found that lateness was never different 
in the program and base-line groups;- that finding did not warrant yearly 
replication. 

, - J- _ 

During the second year's evaluation it was found that the 
attendance of elementary scnool pupils in the program was better than 
that of the base-line group, and that the dropout incidence among high 
school pupils in the program was significantly below that' of the base- 
line group. These results were considered very important because, unlike 
grades and behavior ratings, they were, free of teacher-opinion biases. 
This, and .the fact that they were not obtained in the first year's 
evaluation, made them, worthy- of replication in the third operational year. 
They also represent two of the major objectives of the bilingual program: 

. Participation in the ARRIBA component .will reduce signifi- . 
cantly the amount of pupil absenteeism." (This objective 
was assessed for the elementary school level . ) . 




. Participation at rhe hich school level of the ARRIBA program' 
will reduce i'Sie frequoncy v/ith which pupils drop out of 
schools. (This jbjectivc was assessed for the senior high 
school level.) 

As the study conducted during the second year of prograin opera- 
tion 'vas the rnore conservative of the two assessments of these objectives, 
it was decided that replication of the methodology used at that time would 
be the most conservative way of reaffirming the prograin^s effectiveness . 
As the effectiveness of the prograiu in irriproving grades was already clear, 
it was de'":ided that that study did not need to be replicated. 

Program , 

Tho ARRIBA program served pupils in Grades 3 through 12 • The 
program was geared to pupils whose competence in at least oral Spanish 
was great, due to their having been either native speakers of the language 
or children of households where the parents were native speakers of the 
language. _ The program included academic courses and scire hiah-school-levei 
commercial courses in the Spanish language, as well as English as a Second 
Language. Elementary school pupils and junior high' school students in 
the program. were enrolled in the entire package of courses • High school 
pupils could, with the guidance of their counselor, select any courses 
offered by the program and coufl choose a combination of program classes 
and Bnglish-language courses . 

At the one elementary school, separate classes were operated 
for each grade level. At the second elementary school, pupils in the 
ARRIBA component received instruction in an vingraded classroom. At the 
junior and senior high school levels, pupils followed a prescribed weekly 
program, movinc from teacher to teacher for the various subjects. 

Evaluation ' ' 

Sample . Two samples of pupils were involved in this study ^ 
one at the elementary level and one at the senior high school level.- As 
attendance data were available for virtually every pupil at the elementary 
level, a randrm sample of 25 pupils was drawn from those who had been 
on roll in the program since October , 1971 , and were still on roll at 
the end .o"f the third quarter. The number of days of absence and the per- 
centag^ of attendance were computed for ieach pupil. 

As dropout incidence is . a relatively lov^ base-rate phenomenon, 
all pupils enrolled in the ARRIBA high school component prior, to October 
were drawn for the assessment of the ability of the program, to . keep high 
school- students in school. 

The base-line groups against which thecie samples were compared 
were developed last year and consisted of a 20't si?7nple of all pupils who 
were classed as Spanish-speaking in 1968. Elementary school and senior 
high school subsaraples were used in this study. A detailed description 
of the generation of the base-line group was included in the 1970-1971 
evaluation report.. ' • 




fil. I'.lir: st;Ii(j(jl v<Mir i.iV so.lticr jnq p\ir>:. ;; frorr» Liu,: y^rcijo rt who had bt^t^n on 
roll prior to VAio ortd of Octobor l/)7j and criockinq th*.} attendance record 
of each one Dropout dat^i wore ohtair.od by visitinq the teachers of each 
of the pupils listed iw the pupil-infc^nn^ition file as of October 1971. 
'V\\c tnacherf; * roll books woro cxcuniiied to soe. whether the pupil had beon 
dropped from the class roster, Tf ho had, the school office was con- 
tacted to see whether the pupil .ad transferred, dropped out, or moved. 

Analysis . For last year's evaluation, preprogram base-line 
data on Spanish-speaking pupils were obtained. During the 1971-1972 school 
year, the parallel data were obtained, permitting analysis of variance 
or chi-square to be computed. 

Results 

Table 12.1 ARRIBA Elementary School Program Attendance (First Three 

Quarters of School Year) . 





•Item 


19-7-2-Current 

(N-2.5) 




1968 


-Base-line Group 
(N=77)- 


Mean Percentage of 
Absence 


10.5% 






13.8% 




toalysis of 


Variance 






Source 


: MS 


df 


F" 


P 


Between Groups 
Within Groups 


205.4 
119 . 6 " 


1 

100 


1.7 


NS 



Elementary School Attendance 

Table 1 shows the results of the comparison of ARRIBA elementary 
school pupils* attendance for the 1971-1972 school year with that of the 
base-line group. The ARRIBA . adents were absent an average of 3.3 percen- 
tage points less than the base-line students — a reduction of about one-- 
fifth. This was not a statis^tically significant improvement over the 
base-line group. (In the second year of the program, absenteeism was 
7.2% — a statistically significant change from the . base-line absenteeism. 

High Schools. Dropouts 

Table 12.2 shows the number of pupils on roll prior to October 
31, by grade level, and the number of pupils who dropped out of school 
between October and. March.. The table also shows similar data for the 
1969 base-line group of Spanish-speaking pupils enrolled at the same 
schools u 




175 



enable 12.2 



Comparison of 1971-197;: ■ Dropoxir Incidcnct^ in rhe Iliah School 
ARRIBA component with the 1969 Bai^e-Iine Group (Ocr.ober-to- 
March Interval) . ' 







Grade 


Grade 


Grade 


oj.a.ae 


io uai 


Group 


Statistic 


10 


11 • 


1.2 


Unknown 




ARRIBA Pupils Known 


Total N 


108 


68 


27 


— 


203 


to be on Roll Prior 


Dropout N 


4 


5 






9 


to October 31 


Dropout % 


3.7 


7.4 


— 


— . 


4.4 


ARRIBA Pupils Believed 


Total N 


153 


95 


^ 34 


16 


298 


to be on Roll Prior 


•Dropout N 


13 


9 


4 


1 


25 ■ 


to October 31* 


Dropout % 


8.5 


• 9.5 


8.8 


6,2 


8.4 


Preprogram 


Total N 


143 


88 


• 65 




296 


Base-line Group 


Dropout N 


36 


20 


4 




60 




Dropout % 


25.1 


22. 7 


6.2 




20.0 


Chi~Square (df=l) 




13 .-7 


14.4 


0.2 






("Believed" Group vs. 


Base-line 


«.C1) 


«.01) 


(NS) 







Group) 



*Due to the large amount of missing data, the exact time when 95 
pupils entered the A RRIBA coTiponent is not known. It is known 
that they were in the program before December, 1971. Since most 
of these probably were enrolled in the program in October, they 
are included in this "believed to be on roll" group. 
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Two sets of figures are shov/n. Those for pupils who were lisred 
as proajam participants prior to the beginninc of the Ccrober interval 
are .^hown first. There wero also 95 pupils whose program adirlssio^ dat:os 
wero not available but who were participating by Dece.-nber. Most of these 
were in the prooram prior to October , but it was not known how many were 
adrdtted after this date. The results for the "known" group and the 
"no admission date" group are combined in the second set of data on the 
tcbler "pupils believed on roll.^' As the dropout incidence was higher 
in this combined group r the conservative approach was adopted, and the 
comp. risons with the base line were made with the pupils believed on 
roll by October 31. 

Examination of Table 12.? confirms the findinas of tiie 1970- 
1971 school year: during the October-March interval the dropout incidence 
of the tenth-grade pupils was about one-third that of the base-line 
group; that of eleventh graders was slightly less than half that of the 
basa-line group; among twelf th-rgrade pupils, it was virtually the same 
-as in .the base -line group. 

Discussion 

The findings of this study clearly demonstrate the effectiveness 
of tlie ARRIBA component in preventing dropouts at the high school level. 
The St .dy of absenteeism at the elementary level is not clear: while 
attendance in the observed sample was better than in the base-line group, 
unlike the results froin the second operational year, it was not sufficiently 
different to rule out chance. 

During the second operational year, absenteeism had fallen to 
7.2%, compared with the current year's 10.8%. Both findings suggest that 
more sophisticated studies are desirable. 

The dropout results show th&t on an October-to-March basis the 
program consistently helps to retain students in the schools. Unknown is 
the extent to which the program helps the retention of pupils on a long- 
term basis. Specifically, what percentage of a cohort of pupils who 
have been served by the program in the 10th grade will actually complete 
school and receive diplomas? Of those who do- complete school, are there 
differences in pupils who have participated in project during their 
11th- and 12th grade experiences, and those pupils who have moved into 
regular English-language glasses? This kind of long-range study would 
demonstrate project effectiveness in the most socially m.eaningful way. 
It is, however, a complex and expensive undertaking because of the 
difficulty of following individuals and appropriate comparison groups 
through several years of school experience. 

The elementary school attendance study requires follow up be- 
cause the consistency of the trend suggests that follow-up of a larger 
saicple may be necessary to provide a clear-cut assessL'ent. Tc may also 
be necessary to control family background, grade level, aiid length of 
experience in the program, if unequivocal results are to be obtained. 

In both careas, a^ policy decision needs to be made regarding 
utilization of evaluation resources: haw much is to be spent to better 
establish the. effectiveness of the program in the areas of attendance 
and school retention given the overall, aims of tiie program. 
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